CN113566824A - Vehicle positioning method and device, electronic equipment and storage medium - Google Patents

Abstract

The embodiment of the application discloses a vehicle positioning method, a vehicle positioning device, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring a road image obtained by shooting through a camera device of a vehicle, and identifying a target road sign and a first sign type of the target road sign included in the road image; determining a mark type to be searched corresponding to the first mark type, and acquiring one or more map marks belonging to the mark type to be searched from a map; matching the target road signs with the acquired map signs to obtain matching results of the target road signs; and correcting the positioning position of the vehicle according to the matching result of the target road sign. By implementing the embodiment of the application, the accuracy and the real-time performance of vehicle positioning can be improved.

Description

Vehicle positioning method and device, electronic equipment and storage medium

Technical Field

The present application relates to the field of vehicle technologies, and in particular, to a vehicle positioning method and apparatus, an electronic device, and a storage medium.

Background

Vehicle positioning is the basis for implementing functions such as driver assistance or automatic driving. At present, a vehicle can acquire a Positioning position through a satellite Positioning System such as a Global Positioning System (GPS) and a beidou satellite System. However, satellite positioning signals are easily affected by the environment, and have the problems of weak signal strength, unstable system and the like, thereby causing inaccurate vehicle positioning.

Disclosure of Invention

The embodiment of the application discloses a vehicle positioning method and device, electronic equipment and a storage medium, which can improve the accuracy and real-time performance of vehicle positioning.

The embodiment of the application discloses a vehicle positioning method, which comprises the following steps: acquiring a road image obtained by shooting through a camera device of a vehicle, identifying a target road sign included in the road image and determining a first sign type of the target road sign; determining a mark type to be searched corresponding to the first mark type, and acquiring one or more map marks belonging to the mark type to be searched from a map; matching the target road signs with the acquired map signs to obtain matching results of the target road signs; and correcting the positioning position of the vehicle according to the matching result of the target road sign.

In one embodiment, the type of the flag to be searched includes: the first mark type and a preset second mark type which is easy to be confused with the first mark type.

In one embodiment, the first flag type includes a dashed line type, and the second flag type, which is confusable with the dashed line type, includes a solid line type; or, the first mark type comprises a double solid line type, and the second mark type which is confusable with the double solid line type comprises a single solid line type; alternatively, the first flag type includes a long-dashed line type, and the second flag type, which is confusable with the long-dashed line type, includes a short-dashed line type.

In one embodiment, the matching result comprises: a map location of a target map marker in the map that matches the target road marker; and correcting the positioning position of the vehicle according to the matching result of the target road sign, wherein the correcting comprises the following steps: determining a calculated weight for the target road sign; determining a first conversion matrix corresponding to the current moment according to the calculated weight, the estimated position of the target road sign in a world coordinate system and the map position of the target map sign; the first position is determined according to the road image and the positioning position of the vehicle before correction; and correcting the positioning position of the vehicle by using the first conversion matrix.

In one embodiment, the determining the calculated weight of the target road sign comprises: determining the calculated weight of the target road sign as the calculated weight corresponding to the first sign type.

In one embodiment, the matching result further comprises: one or more matching point pairs of a match between the target road sign and the target map sign; the calculating the weight of the target road sign comprises: calculating the weight of each matching point pair; and, said determining a calculated weight for said target road sign, comprising: determining the distribution position of a first matching point in each matching point pair, wherein the first matching point is a matching point located in the target road sign; and determining the calculation weight of each matching point pair according to the distribution position of the first matching point.

In an embodiment, the matching the target road sign and each acquired map sign to obtain a matching result of the target road sign includes: acquiring a second conversion matrix corresponding to the previous moment, wherein the second conversion matrix is determined according to the matching result of the road sign and the map sign at the previous moment; determining the initial position of the target road sign according to the road image and the positioning position of the vehicle before correction, and correcting the initial position of the target road sign by utilizing the second conversion matrix to obtain the estimated position of the target road sign in a world coordinate system; and matching the target road signs and the map signs according to the estimated positions of the target road signs and the acquired map positions of the map signs in the map to obtain the matching results of the target road signs.

In one embodiment, the vehicle comprises at least two camera devices; and the acquiring of the road image captured by the camera of the vehicle includes: and splicing the images respectively shot by the at least two camera devices to obtain a top view taking the vehicle as the center as a road image.

The embodiment of the application discloses vehicle positioner includes: the identification module is used for acquiring a road image obtained by shooting through a camera device of a vehicle, identifying a target road sign included in the road image and determining a first sign type of the target road sign; the acquisition module is used for determining the type of the mark to be searched corresponding to the first mark type and acquiring one or more map marks belonging to the type of the mark to be searched from a map; the matching module is used for matching the target road sign with each acquired map sign to obtain a matching result of the target road sign; and the correction module is used for correcting the positioning position of the vehicle according to the matching result of the target road sign.

The embodiment of the application discloses an electronic device, which comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor is enabled to realize any vehicle positioning method disclosed by the embodiment of the application.

The embodiment of the application discloses a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program is characterized in that when being executed by a processor, the computer program realizes any one of the vehicle positioning methods disclosed in the embodiment of the application.

Compared with the related art, the embodiment of the application has the following beneficial effects:

the vehicle may acquire a road image captured by the image capturing device, and recognize a target road sign included in the road image and a first sign type of the target road sign. The vehicle may further obtain a type of sign to be looked up corresponding to the first type of sign, and obtain one or more map signs from the map that belong to the type of sign to be looked up. After acquiring the one or more map landmarks, the vehicle may match the target road landmark with each map landmark to obtain a matching result of the target road landmark, so that the positioning position of the vehicle may be corrected using the matching result. The map marks on the map can provide reference information for positioning the vehicle, and are beneficial to correcting the positioning position of the vehicle. In addition, the vehicle does not match the target road sign with all the map signs included in the map, but respectively matches different sign types according to the first sign type of the target road sign, so that the calculation power consumption required by matching is reduced, and the real-time performance of vehicle positioning can be improved. In addition, the target road signs with overlarge type difference can be prevented from being matched with the map signs, so that matching errors can be reduced, and the accuracy of vehicle positioning can be further improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

FIG. 1 is a diagram illustrating an example of an application scenario of a vehicle positioning method according to an embodiment of the disclosure;

FIG. 2 is a schematic method flow diagram of a vehicle location method according to one embodiment;

FIG. 3 is an exemplary illustration of a target road sign, vehicle, and map sign according to one embodiment of the disclosure;

FIG. 4 is a schematic method flow diagram of another vehicle location method disclosed in one embodiment;

FIG. 5 is an exemplary diagram of a target road sign and a target map sign according to one embodiment of the disclosure;

FIG. 6 is a flow diagram illustrating a method for matching target road signs and map signs according to one embodiment;

FIG. 7 is an exemplary illustration of another target road sign, vehicle, and map sign disclosed in one embodiment;

FIG. 8 is a schematic illustration of a vehicle locating device according to an exemplary embodiment;

fig. 9 is a schematic structural diagram of an electronic device disclosed in an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It is to be noted that the terms "comprises" and "comprising" and any variations thereof in the examples and figures of the present application are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.

The embodiment of the application discloses a vehicle positioning method and device, electronic equipment and a storage medium, which can improve the accuracy and real-time performance of vehicle positioning. The following are detailed below.

Referring to fig. 1, fig. 1 is an exemplary diagram of an application scenario of a vehicle positioning method according to an embodiment. As shown in fig. 1, avehicle 10 may be included. Thevehicle 10 may be provided with one or more camera devices, such as a visible light camera, an infrared camera, a fisheye camera, and the like.

Vehicle 10 may travel onroadway 20, which may include one or more different types of roadway signs onroadway 20. The road sign may refer to a graphic symbol for indicating road information such as a lane line, a turn arrow, a deceleration notice line, a zebra crossing, a road sign, etc., but is not limited thereto. As shown in FIG. 1, the road signs on theroad 20 currently being traveled by thevehicle 10 may include a lane line 21, a lane line 22, and aturn arrow 23. As thevehicle 10 travels over theroadway 20, a camera of thevehicle 10 may capture images including one or more roadway signs.

Thevehicle 10 may also obtain amap 30, which may include one or more different types of map markings, which may be digitized versions of road markings on theroads 20 of the real world. As shown in fig. 1, themap 30 may include alane line 31 matching the lane line 21, alane line 32 matching the lane line 22, and a turningarrow 33 matching the turningarrow 23. Themap 30 may include the location of individual map markers, which may include, for example, the absolute coordinates of the map markers in a world coordinate system, and thus themap 30 may provide a reference for vehicle localization.

Referring to fig. 2, fig. 2 is a schematic flow chart of a method for positioning a vehicle according to an embodiment, which can be applied to a vehicle having an on-board computer (ECU), an Electronic Control Unit (ECU), and the like, without limitation. As shown in fig. 2, the method may include:

210. the method includes acquiring a road image captured by a vehicle camera, and identifying a target road sign and a first sign type of the target road sign included in the road image.

In the embodiment of the present application, the position where the vehicle imaging device is disposed may include, but is not limited to, a front windshield, a side mirror, and the like. The road image may be an image captured by a single camera at the current time, or may also be an image obtained by stitching images captured by a plurality of cameras at the current time, which is not limited specifically.

Optionally, when the vehicle includes at least two camera devices, the vehicle may splice images respectively captured by the at least two camera devices at the current time, and obtain a top view taking the vehicle as a center as a road image. When the images are spliced, processing operations related to visual transformation, such as perspective transformation and affine transformation, can be further included. When the road image is a top view with the vehicle as the center, the road image can include road signs of different directions of the vehicle, and can include more information related to the road signs, which is beneficial to improving the success rate of matching the target road signs in the map. For example, when the road image is an image captured by a forward-looking camera of the vehicle, the road image may include only a lane line of a current driving lane of the vehicle; and when the road image is a 360 ° perspective top view of the vehicle, the road image may include a lane line of a lane where the vehicle is currently traveling, and may further include lane lines of other lanes adjacent to the traveling vehicle.

After the vehicle obtains the road image, the target road sign in the road image may be identified. Wherein identifying the target road sign may include: the method includes determining an image position of a target road sign in a road image, and determining a first sign type of the target road sign.

For example, the vehicle may identify a target road sign in the road image based on feature matching. The reference image features of different types of road signs such as lane lines, turning arrows, etc. may be pre-designed. After the road image is acquired, the vehicle may perform feature extraction on the road image, and match the extracted image features to be identified with one or more reference image features. When the image feature to be recognized is matched with the reference image feature, the image position of the target road sign in the road image can be determined according to the image position of the image feature to be recognized in the road image, and the first sign type of the target road sign can be determined as the type of the road sign corresponding to the matched reference image feature.

For example, the vehicle may identify the target road sign in the road image based on an end-to-end approach such as machine learning or deep learning. The recognition model can be pre-constructed, and can be a Neural Network model such as a Convolutional Neural Network (CNN) or a U-Network (U-Net). And training the recognition model by using the sample image carrying the road sign label, so that the recognition model has the recognition capability of the road sign. The road image is input into the trained recognition model, and the image position of the target road sign output by the recognition model in the road image and the first sign type of the target road sign can be obtained.

The embodiment of the present application does not limit the manner of recognizing the target road sign from the road image.

220. And determining the type of the mark to be searched corresponding to the first mark type, and acquiring one or more map marks belonging to the type of the mark to be searched from the map.

In one embodiment, the flag type to be looked up corresponding to the first flag type may comprise the first flag type. That is, the vehicle may obtain one or more map markings of the same marking type as the target road marking from the map.

For example, the target road sign may be a solid lane line, and the vehicle may obtain one or more solid lane lines from the map.

In one embodiment, the flag type to be looked up corresponding to the first flag type may include: a first flag type, and a second flag type that is pre-set and confusable with the first flag type. The second flag type may be set according to actual service requirements, and is not specifically limited. That is, the vehicle may obtain one or more map markers from the map that are of the same marker type as the target road marker, and one or more map markers from the map that are of a marker type that is confusable with the target road marker.

Alternatively, the first flag type may include a dotted line type, and the second flag type, which is confusable with the dotted line type, may include a solid line type. Wherein, the dotted line type may include a single dotted line type and a double dotted line type; the solid line type may include a single solid line type, a double solid line type, but is not limited thereto. For example, if the target road sign is a single-dashed lane line, the vehicle may obtain the single-dashed lane line, the single-solid lane line, and the double-solid lane line from the map.

Alternatively, the first flag type may include a double solid line type, and the second flag type, which is confusable with the double solid line type, may include a single solid line type. For example, if the target road sign is a double-solid lane line, the vehicle may obtain the double-solid lane line and the single-solid lane line from the map.

Alternatively, the first flag type may include a long-dashed line type, and the second flag type, which is confusable with the long-dashed line type, may include a short-dashed line type. The long dotted line type and the short dotted line type are both dotted line types, and the length of the short dotted line type line segment can be shorter than that of the long dotted line type line segment. For example, the dashed line segment used by the lane line of the driving vehicle may be a long dashed line type, and the dashed line segment used by the left-turn waiting area may be a short dashed line type.

It should be noted that, examples of multiple sets of flag types are given above, and each set of flag types includes two or more flag types that are confusable with each other. For example, the implementation type and the dotted line type are a set of confusable flag types, and when a first flag type includes a dotted line type, a second flag type confusable with the dotted line type may include a solid line type; when the first flag type includes a solid line type, the second flag type, which is confusable with the solid line type, may include a dotted line type. The aforementioned double-solid line type and single-solid line type may be a group of confusable flag types, and the long-dashed line type and short-dashed line type may be a group of confusable flag types, which will not be described in detail below.

Due to the influence of factors such as the shooting angle, the image definition and the like, the problem of false recognition may exist when the first sign type of the target road sign is determined according to the road image. If the first flag type is identified incorrectly, map matching errors are likely to result. In the embodiment of the application, the vehicle can be compatible with the identification error of the target road sign by setting the second sign type which is easy to be confused with the first sign type, so that the influence of the type identification error on map matching is reduced, and the overall robustness of the vehicle positioning method is improved.

230. And matching the target road signs with the acquired map signs to obtain the matching results of the target road signs.

After the vehicle acquires one or more map markers with the same marker type as the target road marker or which are easy to be confused from the map, the target road marker and each acquired map marker can be matched.

The target road marking may be the result of imaging of a road marking on the road in the road image, and the map marking on the map may be a digitized version of the road marking on the road. Therefore, matching the target road sign with each map sign may refer to finding out a target map sign that belongs to the same road sign in the real world as the target road sign from the acquired map signs. The manner of matching may include, but is not limited to: nearest neighbor search, Brute-force match (Brute-Froce mather).

For example, if the target road sign is the dotted lane line a, the vehicle may search for a lane line a' matching the lane line a among a plurality of dotted lane lines included in the map. If the target road sign is the turning arrow B, the vehicle may find a lane line B' matching the turning arrow B among a plurality of turning arrows included in the map.

The matching result of the target road sign may include: a target map marker matching the target road marker, and a map location of the target map marker in the map. It should be noted that the map may be constructed based on the world coordinate system, and therefore, the map position of the target map mark in the map may be the position of the target map mark in the world coordinate system.

240. And correcting the positioning position of the vehicle according to the matching result of the target road sign.

The vehicle may determine the relative position of the target road marking with respect to the vehicle based on the image position of the target road marking in the road image. For example, the vehicle may determine the relative position of the target road sign with respect to the camera device according to the image position of the target road sign, and further obtain the relative position of the target road sign with respect to the vehicle according to the external reference of the camera device.

Also, the vehicle may further include a satellite positioning system, a wheel speed meter, an Inertial Measurement Unit (IMU), and the like, which may be used for positioning. The vehicle can acquire the data acquired by the one or more sensors to determine the positioning position of the vehicle in the world coordinate system as the positioning position before correction of the vehicle. Based on the relative position of the target road sign with respect to the vehicle and the preliminarily determined vehicle positioning position, the vehicle may determine an estimated position of the target road sign in the world coordinate system, so that the target road sign and the vehicle may be registered to the map, i.e. the target road sign and the vehicle may be unified to the map.

The vehicle may further determine the first transformation matrix for the current time based on the estimated location of the target road sign in the world coordinate system and a map location of the target map sign that matches the target road sign. The manner of calculating the first transformation matrix may include, but is not limited to, nonlinear Optimization, such as using a General Graph Optimization (G2O) algorithm, or calculation based on a Ceres optimizer.

The first conversion matrix may be used to indicate a difference in position between the estimated position of the target road sign determined by the vehicle based on the uncorrected localized position of the vehicle and the road image and the accurate map position of the target road sign (i.e., the map position of the target map sign). Such a difference may be caused by inaccuracy of the preliminary location position of the vehicle, and thus the location position of the vehicle may be corrected using the first conversion matrix.

Referring to fig. 3, fig. 3 is an exemplary diagram of a target road sign, a vehicle, and a map sign according to an exemplary embodiment. As shown in fig. 3, avehicle 310a may be used to indicate a location preliminarily determined by the vehicle based on one or more sensor data, andlane lines 311a and 312a are target lane markings determined based on the location before correction of the vehicle and a road image. The vehicle 310b may be used to indicate the actual location of the vehicle, withlane line 311b being a target map marker on the map that matcheslane line 311a, andlane line 312b being a target map marker on the map that matcheslane line 312 a. As shown in fig. 3, there is a position difference between thelane line 311a and thelane line 311b, and a position difference between thelane line 312a and thelane line 312b, which can be characterized by the first conversion matrix. Accordingly, if the estimated position of thevehicle 310a is represented as P1_ vec and the first conversion matrix is T1, the corrected vehicle positioning position P2_ vec is P1_ vec × T1, and the corrected P2 is close to the actual vehicle positioning position 301 b.

It should be noted that the vehicle may identify one or more target road signs in the road image. When the road image includes a plurality of target road signs, the vehicle may perform the aforementioned steps 210-230 to obtain a matching result corresponding to each of the plurality of target road signs, so as to correct the positioning position of the vehicle according to the matching results corresponding to the plurality of target road signs.

For example, the vehicle may identify three target road signs from the road image, respectively a dashed lane line a, a turning arrow B, and a solid lane line C. The vehicle can determine the estimated position of the broken lane line a as P1_ a, the estimated position of the steering arrow B as P1_ B, and the estimated position of the solid lane line C as P1_ C, based on the respective target road signs and the vehicle's pre-correction positioning position.

Further, the map position of the broken-line lane line a ' matching the broken-line lane line a on the map may be P2_ a, the map position of the steering arrow B ' matching the steering arrow B on the map may be P2_ B, and the map position of the solid-line lane line C ' matching the solid-line lane line C on the map may be P2_ C.

The vehicle may determine the first conversion matrix at the current time from P1_ A, P2_ a corresponding to the broken line lane line a, P1_ B, P2_ B corresponding to the steering arrow B, and P1_ C, P2_ C corresponding to the solid line lane line C, thereby correcting the location position of the vehicle using the first conversion matrix at the current time.

Therefore, in the foregoing embodiment, the vehicle may capture an image of the target road sign on the road through the image capturing device, match the target road sign with the map sign in the map, and correct the positioning position of the vehicle by using the reference information provided by the map sign on the map, so as to improve the accuracy of vehicle positioning. In addition, the vehicle does not match the target road sign with all map signs included in the map, but determines the corresponding type of the sign to be searched according to the first sign type of the target road sign, so that the map sign belonging to the type of the sign to be searched is obtained in the map, and the target road sign is matched with the map sign belonging to the type of the sign to be searched in the map. That is to say, the vehicle can be matched with different mark types separately, which is beneficial to reducing the calculation power consumption required by matching and can improve the real-time performance of vehicle positioning. In addition, the target road signs with overlarge type difference can be prevented from being matched with the map signs, so that matching errors can be reduced, and the accuracy of vehicle positioning can be further improved.

Referring to fig. 4, fig. 4 is a flowchart illustrating another method for positioning a vehicle according to an embodiment of the present disclosure. As shown in fig. 4, the method may include:

410. a road image captured by a camera of a vehicle is acquired, a target road sign included in the road image is identified, and a first sign type of the target road sign is determined.

420. And determining the type of the mark to be searched corresponding to the first mark type, and acquiring one or more map marks belonging to the type of the mark to be searched from the map.

The vehicle can determine the first mark type as the mark type to be searched; alternatively, the first flag type and the second flag type that is confusable with the first flag type may be determined as the flag type to be searched.

430. And matching the target road signs with the acquired map signs to obtain the matching results of the target road signs.

The implementation of steps 410-430 can be seen in the foregoing embodiments, and the following description is omitted.

440. The calculated weight of the target road sign is determined.

The calculated weight of the target road sign may refer to the weight when the first conversion matrix is calculated. The vehicle may identify one or more target road signs in the road image, each target road sign may correspond to a calculation weight, and the calculation weights of different target road signs may be the same or different, and are not limited specifically.

Alternatively, the vehicle may determine the calculated weight based on the first sign type of the target road sign. The calculation weights respectively corresponding to the road signs of different sign types may be preset, and after the first sign type is determined, the vehicle may determine the calculation weight of the target road sign as the calculation weight corresponding to the first sign type. The calculation weight of the target road sign is determined according to the sign type, the importance degree of the road signs of different sign types in calculating the conversion matrix can be adjusted based on the characteristics of different sign types, the calculation accuracy of the conversion matrix is improved, and therefore the accuracy of vehicle positioning can be improved.

For example, the computed weight for a feature-rich token type may be higher than the computed weight for a feature-simple token type. When the image features of the road signs with different sign types are extracted, the number of the image features extracted by the sign types with rich features can be more than that of the image features extracted by the sign types with simple features. For example, a turn arrow, a dashed line type, etc. may be a feature-rich flag type, and a solid line type may be a feature-simple flag type. Therefore, the calculation weight corresponding to the turning arrow, the dotted line type may be higher than that corresponding to the solid line type. The mark types with rich characteristics have more extracted image characteristics, so that the error probability in image identification and map matching is relatively lower than that of the mark types with simple characteristics. Therefore, higher calculation weight is given to the mark types with rich characteristics, which is beneficial to improving the calculation accuracy of the transformation matrix, thereby improving the accuracy of vehicle positioning.

Alternatively, if the matching result includes one or more matching point pairs matching between the target road sign and the target map sign, the calculated weight of the target road sign may include the calculated weight of each matching point pair. The vehicle determining the calculated weight of the target road sign may include, but is not limited to, the following:

the first method is as follows: the calculated weight of each matching point pair is determined as the calculated weight corresponding to the first token type.

The second method comprises the following steps: determining the distribution position of a first matching point in each matching point pair, wherein the first matching point is a matching point located in a target road sign; and determining the calculation weight of each matching point pair according to the distribution position of the first matching point.

The calculation weights corresponding to different distribution positions of the first matching point may be preset, and the calculation weights corresponding to different distribution positions may be the same or different, and are not specifically limited. When the vehicle determines the calculation weight of each matching point pair according to the distribution position of the first matching point, the calculation weight of each matching point pair may be directly determined as the calculation weight corresponding to the distribution position of the first matching point. For example, the calculated weight corresponding to the end point of the target road sign at the distribution position may be set higher than the calculated weight corresponding to the middle area of the target road sign.

Or, when the vehicle determines the calculation weight of each matching point pair according to the distribution position of the first matching point, an initial calculation weight may be set for each matching pair, and then the initial calculation weight may be adjusted according to the distribution position of the first matching point. The initial calculation weight may be a random default value, or the initial calculation weight may also be a calculation weight corresponding to the first sign type of the target road sign, which is not limited in particular. For example, when the initial calculation weight is adjusted according to the distribution position, if the distribution position of the first matching point is the end point of the target road sign, the calculation weight of the matching point pair may be increased on the basis of the initial calculation weight; if the allocation position of the first matching point is in the middle area of the target road sign, the calculation weight of the matching point pair may be reduced on the basis of the initial calculation weight. That is, the calculated weight of the matching point pairs at the end points is higher than the calculated weight of the matching point pairs at the middle region.

The vehicle determines the calculation weight of the matching point pair according to different distribution positions of the first matching point, and the importance degree of each matching pair in calculating the first conversion matrix can be adjusted by using the characteristics of different distribution positions, so that the calculation accuracy of the first conversion matrix is improved, and the accuracy of vehicle positioning is improved.

Referring to fig. 5, fig. 5 is an exemplary diagram of a target road sign and a target map sign according to an embodiment of the disclosure. As shown in fig. 5, thetarget road sign 510a is recognized from the road image and may be mapped onto a map according to the localization position of the vehicle before correction, and thetarget map sign 510b is a map sign on the map that matches thetarget road sign 510 a.

The matching result of thetarget road sign 510a and thetarget map sign 510b may include a plurality of matching pairs, and each matching pair may include a first matching point located on thetarget road sign 510a and a second matching point located on thetarget map sign 510 b. As shown in fig. 5, thefirst matching point 511a and thesecond matching point 511b may constitute a first matching point pair, thefirst matching point 513a and thesecond matching point 513b may constitute a third matching point pair, and thefirst matching point 515a and thesecond matching point 515b may constitute a fifth matching point pair.

As shown in fig. 5, different pairs of matching points have different distribution positions on thetarget road sign 510a and thetarget map sign 510b, and two matching points included in the same pair of matching points have the same distribution positions on thetarget road sign 510a and thetarget map sign 510b, respectively.

The distribution positions of thefirst matching point 511a in the first matching point pair and thefirst matching point 515a in the fifth matching point pair may be end points of thetarget road sign 510 a. Wherein, based on the driving direction of the vehicle, the distribution position of thefirst matching point 511a in the first matching point pair may be the top end of thetarget road sign 510 a; the distribution position of thefirst matching point 515a in the fifth pair of matching points may be the bottom end of thetarget road sign 510 a.

The distribution position of thefirst matching point 513a in the third matching point pair may be the middle of thetarget road sign 510 a.

If the calculation weight of the matching point pair at the end point is higher than that of the matching point pair in the middle area, the calculation weight of the first matching point pair and the fifth matching point pair is higher than that of the third matching point pair when the vehicle calculates the first conversion matrix corresponding to the current time.

450. And determining a first conversion matrix corresponding to the current moment according to the calculated weight of the target road sign, the estimated position of the target road sign in the world coordinate system and the map position of the target map sign.

The calculated weight of the target road sign can be used for adjusting the importance degree of road signs of different sign types when calculating the first conversion matrix; or, the method can also be used for adjusting the importance degree of the matching points at different distribution positions in the same target road sign during calculation of the first conversion matrix, and is beneficial to improving the calculation accuracy of the first conversion matrix, thereby improving the accuracy of vehicle positioning.

It should be noted that, if the matching result includes one or more matching point pairs matching between the target road sign and the target map sign, the estimated position of the target road sign in the world coordinate system may include: the estimated position of the first matching point in each matching point pair in a world coordinate system; the map location of the target map marker may include: the map location of the second match point in each pair of match points in the map.

Therefore, when the vehicle determines the first conversion matrix corresponding to the current time, the first conversion matrix corresponding to the current time can be determined according to the calculated weight of each matching point pair, the estimated position of the first matching point in each matching point pair and the map position of the second matching point in each matching point pair.

460. And correcting the positioning position of the vehicle by using the first conversion matrix.

It can be seen that, in the foregoing embodiment, the vehicle may search for matching on the map separately for different mark types, and may also give respective corresponding calculation weights to different mark types when calculating the first conversion matrix at the current time, so that the calculation amount of search for matching may be reduced, and the calculation accuracy of the conversion matrix may be improved, thereby improving the accuracy of vehicle positioning.

As can be seen from the foregoing embodiments, the accuracy of the matching result of the target road sign may affect the accuracy of vehicle positioning. Referring to fig. 6, fig. 6 is a flowchart illustrating a method for matching a target road sign and a map sign according to an embodiment. That is, the step shown in fig. 6 may be an implementation of thestep 230 or thestep 430. As shown in fig. 6, the following steps may be included:

610. and acquiring a second conversion matrix corresponding to the previous moment, wherein the second conversion matrix is determined according to the matching result of the road mark and the map mark at the previous moment.

The vehicle moves continuously during traveling, and therefore the same road sign exists in road images captured at different times. Based on this characteristic, when the vehicle corrects the vehicle positioning position at the present time, the information acquired at the previous time can be reused.

For example, after the vehicle determines the location position of the vehicle before correction by satellite positioning or inertial navigation, etc., a search window may be constructed on the map with the location position of the vehicle before correction as the center, and a map mark belonging to the type of the mark to be searched may be obtained from the search window. The search window may move as the vehicle moves during the travel of the vehicle.

620. And determining the initial position of the target road sign at the current moment according to the road image at the current moment and the positioning position of the vehicle before correction.

The vehicle may determine a relative position between the target road marking and the vehicle based on the image position of the target road marking in the road image at the current time, and determine an initial position of the target road marking at the current time based on the relative position and an uncorrected localized position of the vehicle at the current time.

630. And correcting the initial position of the target road sign by using the second conversion matrix corresponding to the previous moment to obtain the estimated position of the target road sign in the world coordinate system.

For example, the initial position of the target road sign at the current time may be P1_ sym, the second transformation matrix may be T2, and the estimated position P2_ sym of the target road sign in the world coordinate system is P1_ sym × T2.

640. And matching the target road signs and the map signs according to the estimated positions of the target road signs and the map positions of the map signs in the map, which belong to the signs to be searched, on the map to obtain the matching results of the target road signs.

The vehicle can determine the distance between each map mark and the target road mark according to the map position of each map mark and the estimated position of the target road mark, and determine the map mark with the shortest distance as the target map mark matched with the target road mark.

After the initial position of the target road sign is corrected by the vehicle by using the second conversion matrix, the distance between the corrected target road sign and the matched target map sign is closer, and the vehicle can accurately and quickly find out the target map sign from the map, thereby being beneficial to improving the accuracy of the matching result of the target road sign.

Referring to fig. 7, fig. 7 is an exemplary illustration of another exemplary target road sign, vehicle, and map sign disclosed in one embodiment. As shown in fig. 7, thevehicle 710 may be an uncorrected positioning position of the vehicle at the current time, and thevehicle 720 may be an uncorrected positioning position of the vehicle at the previous time. The target road sign identified from the road image at the previous time may be thelane line 721 a. The target map flag on the map that matches thelane line 721a may be thelane line 730. At the previous time, the vehicle may determine the second transformation matrix at the previous time based on the estimated location of thelane line 721a, the map location of thelane line 730.

The initial position of the target road sign determined by the vehicle based on the image position of the target road sign in the road image and the positioning position of the vehicle before correction at the current time may be shown as alane line 722a in fig. 7. The vehicle does not change the lane of travel at the present time compared to the previous time, and thus the target map sign actually matching the target road sign should be thelane line 720 in fig. 7.

However, if thelane line 722a is similar to the distance between thelane line 730 and thelane line 740 on the map, and thelane line 722a is directly matched with each map mark on the map, thelane line 740 may be erroneously recognized as a target map mark matched with the target road mark, which may cause an error in calculation of the first conversion matrix.

If the initial position of the target road sign is corrected using the second transformation matrix at the previous time, the corrected estimated position of the target road sign may be as shown by thelane line 722b in fig. 7. Thelane line 722b is closer to thelane line 730 than thelane line 740, and thus the probability that the vehicle determines thelane line 730 as the target map marking that matches the target road marking is higher.

Therefore, the initial position of the target road sign at the current moment is corrected by using the second conversion matrix at the previous moment, so that a more accurate estimated position can be provided for the target road sign before matching and searching, the accuracy of the matching result of the target road sign is improved, and the accuracy of the corrected vehicle positioning can be improved.

Referring to fig. 8, fig. 8 is a schematic structural diagram of a vehicle positioning device according to an embodiment of the disclosure, where the vehicle positioning device may be disposed on the vehicle. As shown in fig. 8, thevehicle positioning apparatus 800 may include: anidentification module 810, anacquisition module 820, amatching module 830, and acorrection module 840.

Theidentification module 810 is configured to acquire a road image captured by a camera of a vehicle, and identify a target road sign and a first sign type of the target road sign included in the road image;

an obtainingmodule 820, configured to determine a to-be-searched flag type corresponding to the first flag type, and obtain one or more map flags belonging to the to-be-searched flag type from a map;

thematching module 830 is configured to match the target road sign with each acquired map sign to obtain a matching result of the target road sign;

and the correctingmodule 840 is used for correcting the positioning position of the vehicle according to the matching result of the target road sign.

In one embodiment, the type of flag to be looked up may include: a first flag type, and a second flag type that is pre-set and confusable with the first flag type.

In one embodiment, the first flag type may include a dashed line type, and the second flag type, which is confusable with the dashed line type, may include a solid line type; or,

the first flag type may include a double solid line type, and the second flag type, which is confusable with the double solid line type, may include a single solid line type or a double dotted line type; or,

the first flag type may include a long-dashed line type, and the second flag type, which is confusable with the long-dashed line type, may include a short-dashed line type.

In one embodiment, the matching result of the target road sign may include: the target map markings match the target road markings at the map location in the map.

Themodification module 840 may include: weight determining unit, matrix determining unit and positioning correcting unit.

A weight determination unit operable to determine a calculated weight of the target road sign;

the matrix determining unit is used for determining a first conversion matrix corresponding to the current moment according to the calculated weight, the estimated position of the target road sign in the world coordinate system and the map position of the target map sign; the first position is determined according to the road image and the positioning position of the vehicle before correction;

and the positioning correction unit can be used for correcting the positioning position of the vehicle by using the first conversion matrix.

In one embodiment, the weight determination unit may be further configured to determine the calculated weight of the target road sign as the calculated weight corresponding to the first sign type.

In one embodiment, the matching result of the target road sign may further include: one or more matching point pairs matching between the target road marking and the target map marking; the calculating the weight of the target road sign includes: calculating the weight of each matching point pair;

the weight determining unit can be further used for determining the distribution position of a first matching point in each matching point pair, wherein the first matching point is a matching point located on a target road sign; and determining the calculation weight of each matching point pair according to the distribution position of the first matching point.

In one embodiment, thematching module 830 may include: the device comprises an acquisition unit, a position correction unit and a matching unit.

The acquisition unit is used for acquiring a second conversion matrix corresponding to the previous moment, and the second conversion matrix is determined according to the matching result of the road mark and the map mark at the previous moment;

the position correction unit can be used for determining the initial position of the target road sign according to the road image and the positioning position of the vehicle before correction, and correcting the initial position of the target road sign by utilizing a second conversion matrix to obtain the estimated position of the target road sign in the world coordinate system;

and the matching unit can be used for matching the target road signs and the map signs according to the estimated positions of the target road signs and the acquired map positions of the map signs in the map to obtain the matching results of the target road signs.

In one embodiment, a vehicle provided with thevehicle localization apparatus 800 may include at least two camera devices. And, thevehicle positioning apparatus 800 may further include: and (5) splicing the modules.

And the splicing module can be used for splicing the images respectively shot by the at least two camera devices to obtain a top view taking the vehicle as the center as a road image.

It can be seen that, in the foregoing embodiment, the vehicle positioning device may capture an image of a target road sign on a road through the image capturing device, and match the target road sign with a map sign in a map, so that the positioning position of the vehicle may be corrected by using reference information provided by the map sign on the map, and the accuracy of vehicle positioning may be improved. In addition, the vehicle positioning device can be used for separating different mark types for respective matching, so that the calculation power consumption required by matching is reduced, and the real-time performance of vehicle positioning can be improved. In addition, the target road signs with overlarge type difference can be prevented from being matched with the map signs, so that matching errors can be reduced, and the accuracy of vehicle positioning can be further improved.

Referring to fig. 9, fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.

As shown in fig. 9, theelectronic device 900 may include:

amemory 910 storing executable program code;

aprocessor 920 coupled with thememory 910;

theprocessor 920 calls the executable program code stored in the memory 9101 to execute any one of the vehicle positioning methods disclosed in the embodiments of the present application.

It should be noted that the electronic device shown in fig. 9 may further include components, which are not shown, such as a power supply, an input key, a camera, a speaker, a screen, an RF circuit, a Wi-Fi module, a bluetooth module, and a sensor, which are not described in detail in this embodiment.

The embodiment of the application discloses a computer readable storage medium which stores a computer program, wherein when the computer program is executed by a processor, the processor is enabled to realize any one of the vehicle positioning methods disclosed in the embodiment of the application.

Embodiments of the present application disclose a computer program product comprising a non-transitory computer readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform any one of the vehicle localization methods disclosed in the embodiments of the present application.

It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. Those skilled in the art should also appreciate that the embodiments described in this specification are all alternative embodiments and that the acts and modules involved are not necessarily required for this application.

In various embodiments of the present application, it should be understood that the size of the serial number of each process described above does not mean that the execution sequence is necessarily sequential, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated units, if implemented as software functional units and sold or used as a stand-alone product, may be stored in a computer accessible memory. Based on such understanding, the technical solution of the present application, which is a part of or contributes to the prior art in essence, or all or part of the technical solution, may be embodied in the form of a software product, stored in a memory, including several requests for causing a computer device (which may be a personal computer, a server, a network device, or the like, and may specifically be a processor in the computer device) to execute part or all of the steps of the above-described method of the embodiments of the present application.

It will be understood by those skilled in the art that all or part of the steps in the methods of the embodiments described above may be implemented by hardware instructions of a program, and the program may be stored in a computer-readable storage medium, where the storage medium includes Read-Only Memory (ROM), Random Access Memory (RAM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), One-time Programmable Read-Only Memory (OTPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Compact Disc Read-Only Memory (CD-ROM), or other Memory, such as a magnetic disk, or a combination thereof, A tape memory, or any other medium readable by a computer that can be used to carry or store data.

The vehicle positioning method, the vehicle positioning apparatus, the electronic device, and the storage medium disclosed in the embodiments of the present application are described in detail above, and specific examples are applied herein to illustrate the principles and implementations of the present application, and the descriptions of the above embodiments are only used to help understand the method and the core ideas of the present application. Meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims (11)

1. A vehicle positioning method, characterized in that the method comprises:

acquiring a road image obtained by shooting through a camera device of a vehicle, and identifying a target road sign and a first sign type of the target road sign included in the road image;

determining a mark type to be searched corresponding to the first mark type, and acquiring one or more map marks belonging to the mark type to be searched from a map;

matching the target road signs with the acquired map signs to obtain matching results of the target road signs;

and correcting the positioning position of the vehicle according to the matching result of the target road sign.

2. The method of claim 1, wherein the type of flag to be looked up comprises: the first mark type and a preset second mark type which is easy to be confused with the first mark type.

3. The method of claim 2, wherein:

the first flag type includes a dotted line type, and the second flag type, which is confusable with the dotted line type, includes a solid line type; or,

the first flag type includes a double solid line type, and the second flag type, which is confusable with the double solid line type, includes a single solid line type; or,

the first flag type includes a long-dashed line type, and the second flag type, which is confusable with the long-dashed line type, includes a short-dashed line type.

4. The method of claim 1, wherein the matching result comprises: a map location of a target map marker in the map that matches the target road marker; and correcting the positioning position of the vehicle according to the matching result of the target road sign, wherein the correcting comprises the following steps:

determining a calculated weight for the target road sign;

determining a first conversion matrix corresponding to the current moment according to the calculated weight, the estimated position of the target road sign in a world coordinate system and the map position of the target map sign; the first position is determined according to the road image and the positioning position of the vehicle before correction;

and correcting the positioning position of the vehicle by using the first conversion matrix.

5. The method of claim 4, wherein determining the calculated weight of the target road sign comprises:

determining the calculated weight of the target road sign as the calculated weight corresponding to the first sign type.

6. The method of claim 4, wherein the matching result further comprises: one or more matching point pairs of a match between the target road sign and the target map sign; the calculating the weight of the target road sign comprises: calculating the weight of each matching point pair; and, said determining a calculated weight for said target road sign, comprising:

determining the distribution position of a first matching point in each matching point pair, wherein the first matching point is a matching point located in the target road sign;

and determining the calculation weight of each matching point pair according to the distribution position of the first matching point.

7. The method according to claim 1, wherein the matching the target road sign and the obtained map signs to obtain the matching result of the target road sign comprises:

acquiring a second conversion matrix corresponding to the previous moment, wherein the second conversion matrix is determined according to the matching result of the road sign and the map sign at the previous moment;

determining the initial position of the target road sign according to the road image and the positioning position of the vehicle before correction, and correcting the initial position of the target road sign by utilizing the second conversion matrix to obtain the estimated position of the target road sign in a world coordinate system;

and matching the target road signs and the map signs according to the estimated positions of the target road signs and the acquired map positions of the map signs in the map to obtain the matching results of the target road signs.

8. The method of any of claims 1-7, wherein the vehicle comprises at least two camera devices; and the acquiring of the road image captured by the camera of the vehicle includes:

and splicing the images respectively shot by the at least two camera devices to obtain a top view taking the vehicle as the center as a road image.

9. A vehicle positioning device, comprising:

the identification module is used for acquiring a road image obtained by shooting through a camera device of a vehicle, and identifying a target road sign and a first sign type of the target road sign included in the road image;

the acquisition module is used for determining the type of the mark to be searched corresponding to the first mark type and acquiring one or more map marks belonging to the type of the mark to be searched from a map;

the matching module is used for matching the target road sign with each acquired map sign to obtain a matching result of the target road sign;

and the correction module is used for correcting the positioning position of the vehicle according to the matching result of the target road sign.

10. An electronic device comprising a memory and a processor, the memory having stored thereon a computer program that, when executed by the processor, causes the processor to implement the method of any one of claims 1 to 8.

11. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1 to 8.

Images