CN103386975A - Vehicle obstacle avoidance method and system based on machine vision - Google Patents

Abstract

Translated fromChinese

本发明提出了一种基于机器视觉的车辆避障方法及系统，该避障方法包括如下步骤：处理器采用基于单幅图像的障碍物检测算法判定缩微车前方是否存在障碍物；处理器对两个摄像头进行标定，采用立体视觉的方法确定障碍物的高度并将高度信息传输给控制器；测距装置检测相邻两车道的车辆状况，为避障换道提供可行驶的区域并将可行使区域的信息传输给控制器；控制器根据获得的信息，采用自适应换道策略，向运行控制模块发送命令，完成车辆的自主换道。本发明的避障方法稳定、自适应程度高，车辆避障姿态流畅，避障成功率高达98%以上。

The present invention proposes a vehicle obstacle avoidance method and system based on machine vision. The obstacle avoidance method includes the following steps: the processor uses an obstacle detection algorithm based on a single image to determine whether there is an obstacle in front of the miniature vehicle; The camera is calibrated, and the height of the obstacle is determined by the method of stereo vision and the height information is transmitted to the controller; the distance measuring device detects the vehicle condition of the adjacent two lanes, provides a driving area for obstacle avoidance and lane change, and can be used for driving. The information of the area is transmitted to the controller; the controller adopts an adaptive lane-changing strategy based on the obtained information, and sends commands to the operation control module to complete the autonomous lane-changing of the vehicle. The obstacle avoidance method of the present invention is stable and has a high degree of self-adaptation, the obstacle avoidance posture of the vehicle is smooth, and the obstacle avoidance success rate is as high as 98%.

Description

Translated fromChinese

一种基于机器视觉的车辆避障方法及系统A vehicle obstacle avoidance method and system based on machine vision

技术领域technical field

本发明涉及智能交通技术领域，具体涉及一种基于机器视觉的车辆避障方法及系统。The invention relates to the technical field of intelligent transportation, in particular to a machine vision-based vehicle obstacle avoidance method and system.

背景技术Background technique

随着城市化进程的加快，现有的交通基础设施和管理办法落后于时代的发展。单纯通过拓宽道路、建设高架、铺设轨道交通、设立标志、鼓励乘坐公共交通、甚至空中交通，依靠传统的方法远远不能适应现代交通的发展，需要发展智能交通系统。智能交通系统是将先进的信息技术、数据通信传输技术、控制技术以及人工智能技术等有效地综合运用于整个交通管理体系而建立起来的大范围、全方位发挥作用的实时、准确、高效运输的综合交通指挥、管理与控制系统。With the acceleration of urbanization, the existing transportation infrastructure and management methods lag behind the development of the times. Simply by widening roads, building elevated roads, laying rail transit, setting up signs, encouraging public transportation, and even air traffic, relying on traditional methods is far from being able to adapt to the development of modern transportation, and requires the development of intelligent transportation systems. The intelligent transportation system is a real-time, accurate and efficient transportation system with a large scope and full range of functions established by effectively comprehensively applying advanced information technology, data communication transmission technology, control technology and artificial intelligence technology to the entire traffic management system. Integrated traffic command, management and control system.

智能交通系统产生于上世纪60年代末。上世纪80年代以来，该领域的研究进入了一个飞速发展的阶段。美国、日本、加拿大、德国、法国等西方主要经济强国都对此投入大量人力物力，可以说“智能交通”是交通运输进入信息时代的重要标志。将先进的智能交通系统应用于现有交通设施，可以有效减少交通负荷和环境污染、保证交通安全、提高运输效率、促进社会经济发展、提高人民生活质量，并能够推动社会信息化及新产业的形成。更重要的是，随着现代化技术的不断前进，也使得交通智能化有了实现的可能。作为未来交通发展的趋势之一，我国政府及科技、交通管理等有关部门高度重视并积极推动智能交通系统的发展。智能交通系统在中国的开发和应用将会形成一个巨大的市场，规模可在百亿甚至千亿元以上，它势必对中国的道路、交通、通讯、电信、交通管理等各个方面产生巨大的推动。Intelligent transportation system was produced in the late 1960s. Since the 1980s, research in this field has entered a stage of rapid development. The United States, Japan, Canada, Germany, France and other major western economic powers have invested a lot of manpower and material resources in this regard. It can be said that "smart transportation" is an important symbol of transportation entering the information age. Applying advanced intelligent transportation systems to existing transportation facilities can effectively reduce traffic load and environmental pollution, ensure traffic safety, improve transportation efficiency, promote social and economic development, improve people's quality of life, and promote social informatization and new industries. form. More importantly, with the continuous advancement of modern technology, it is also possible to realize intelligent transportation. As one of the future trends of transportation development, the Chinese government and relevant departments such as science and technology and traffic management attach great importance to and actively promote the development of intelligent transportation systems. The development and application of intelligent transportation systems in China will form a huge market, with a scale of more than 10 billion or even 100 billion yuan. It is bound to give a huge boost to China's roads, transportation, communications, telecommunications, traffic management and other aspects .

作为智能交通系统的重要研究内容之一，智能车辆驾驶主要研究无人驾驶技术或者作为辅助驾驶系统帮助驾驶员完成车辆驾驶任务。这些任务包括跟踪道路，保持车辆行驶在正确的道路上，维持车辆之间的一个安全距离，根据当前的交通状况和道路特征调节车辆的速度，横跨车道以达到超车和避障的目的以及找到达目的地的最短路径和在市区内方便的行驶和停靠。智能车辆驾驶系统集中地运用了计算机、传感器、信息融合、通讯、人工智能以及自动控制等技术，是典型的高新技术综合体。智能车辆驾驶系统将有效减轻驾驶员的负担，减少驾驶员疲劳驾驶的现象，有利于提高交通安全，同时，配合城市交通控制系统，合理分配交通流，实现交通顺畅。随着计算机和机器人技术的飞速发展，智能车辆研究已经取得了长足进展，并广泛应用于军事、科研、民用等各个领域。在军事方面，智能车辆可以在危险地带代替士兵完成战场侦察等任务；在科研方面，智能车辆可以在外星从事勘探等工作；在民用方面，可作为自动或辅助驾驶系统来减少交通事故。As one of the important research contents of intelligent transportation system, intelligent vehicle driving mainly studies unmanned driving technology or assists drivers to complete vehicle driving tasks as an auxiliary driving system. These tasks include following the road, keeping the vehicle on the correct road, maintaining a safe distance between vehicles, adjusting the speed of the vehicle according to the current traffic conditions and road characteristics, crossing the lane to achieve the purpose of overtaking and avoiding obstacles, and finding The shortest route to the destination and convenient driving and stopping in the urban area. The intelligent vehicle driving system intensively uses technologies such as computers, sensors, information fusion, communication, artificial intelligence and automatic control, and is a typical high-tech complex. The intelligent vehicle driving system will effectively reduce the driver's burden and reduce the driver's fatigue driving phenomenon, which is conducive to improving traffic safety. At the same time, it cooperates with the urban traffic control system to rationally distribute traffic flow and achieve smooth traffic. With the rapid development of computer and robot technology, intelligent vehicle research has made great progress, and it is widely used in military, scientific research, civilian and other fields. In terms of military, smart vehicles can replace soldiers in dangerous areas to complete tasks such as battlefield reconnaissance; in terms of scientific research, smart vehicles can perform tasks such as exploration on alien planets; in terms of civilian use, they can be used as automatic or assisted driving systems to reduce traffic accidents.

视觉系统在智能车辆系统中主要起到环境探测和辨识的作用。与其他的传感器相比，计算机视觉具有检测信息量大、能够遥测等优点。缺点是在复杂环境下，要将探测的目标与背景区分开，将有用信息提取出来所需的计算量很大，单纯以硬件条件来解决，容易导致系统的实时性较差。目前的智能车辆技术中自主导航和自动驾驶是智能车辆开发的关键技术，而自主导航和自动驾驶的实现过程中，非常关键的技术是完成障碍物的识别和避障，这是一个亟需解决的技术问题。The vision system mainly plays the role of environment detection and recognition in the intelligent vehicle system. Compared with other sensors, computer vision has the advantages of large amount of detection information and telemetry. The disadvantage is that in a complex environment, it is necessary to distinguish the detected target from the background, and the calculation required to extract useful information is very large, and it is easy to lead to poor real-time performance of the system if it is only solved by hardware conditions. In the current intelligent vehicle technology, autonomous navigation and automatic driving are the key technologies for the development of intelligent vehicles. In the process of realizing autonomous navigation and automatic driving, the very key technology is to complete the identification and avoidance of obstacles. This is an urgent problem to be solved. technical problems.

发明内容Contents of the invention

为了克服上述现有技术中存在的缺陷，本发明的目的是提供一种基于机器视觉的车辆避障方法及系统，提高车辆行驶过程中的避障成功率、车辆行驶的稳定性更强。In order to overcome the above-mentioned defects in the prior art, the object of the present invention is to provide a vehicle obstacle avoidance method and system based on machine vision, which can improve the success rate of obstacle avoidance during vehicle driving and enhance the stability of vehicle driving.

为了实现本发明的上述目的，根据本发明的一个方面，本发明提供了一种基于机器视觉的车辆避障方法，包括如下步骤：In order to achieve the above object of the present invention, according to one aspect of the present invention, the present invention provides a vehicle obstacle avoidance method based on machine vision, comprising the steps of:

S1：处理器采用基于单幅图像的障碍物检测算法判定缩微车前方是否存在障碍物，如果有障碍物，则将障碍物距离和障碍物所处车道的检测信息传输给控制器，并执行步骤S2，如果没有障碍物，则继续执行步骤S1；S1: The processor uses an obstacle detection algorithm based on a single image to determine whether there is an obstacle in front of the miniature car. If there is an obstacle, it transmits the detection information of the obstacle distance and the lane where the obstacle is located to the controller, and executes the steps S2, if there is no obstacle, proceed to step S1;

S2：处理器对两个摄像头进行标定，采用立体视觉的方法判定障碍物中心位置、上边界中心位置的三维坐标，从而确定障碍物的高度并将所述障碍物高度信息传输给控制器；S2: The processor calibrates the two cameras, and uses stereo vision to determine the three-dimensional coordinates of the center position of the obstacle and the center position of the upper boundary, so as to determine the height of the obstacle and transmit the height information of the obstacle to the controller;

S3：车身两侧的测距装置检测相邻两车道的车辆状况，为避障换道提供可行驶的区域并将所述可行使区域的信息传输给所述控制器；S3: The distance measuring devices on both sides of the vehicle body detect the vehicle conditions of the adjacent two lanes, provide a drivable area for obstacle avoidance and lane change, and transmit the information of the drivable area to the controller;

S4：控制器根据获得的障碍物距离、障碍物所处车道、障碍物高度以及相邻两车道的可行使区域信息，采用自适应换道策略，向运行控制模块发送命令，控制车轮的转向和速度，完成车辆的自主换道。S4: The controller adopts an adaptive lane-changing strategy based on the obtained obstacle distance, the lane where the obstacle is located, the height of the obstacle, and the maneuverable area information of the two adjacent lanes, and sends commands to the operation control module to control the steering and steering of the wheels. speed to complete the autonomous lane change of the vehicle.

在本发明的一种优选实施方式中，所述单幅图像的障碍物检测算法的步骤为：In a preferred embodiment of the present invention, the steps of the obstacle detection algorithm of the single image are:

S21：对图像进行色彩空间变换RGB转HSV，并单独抽取S通道图像；S21: Perform color space transformation on the image from RGB to HSV, and separately extract the S channel image;

S22：对S图像利用大津法进行动态阈值二值化得到二值图像Bin，对Bin图像去噪，消除干扰；S22: Perform dynamic threshold binarization on the S image using the Otsu method to obtain a binary image Bin, denoise the Bin image, and eliminate interference;

S23：在Bin图像上搜索所有可能存在的障碍物轮廓，根据设置的筛选条件将误判的障碍物剔除；S23: Search for all possible obstacle contours on the Bin image, and remove misjudged obstacles according to the set filter conditions;

S24：并根据车道线的方位判定障碍物所处车道。S24: and determine the lane where the obstacle is located according to the orientation of the lane line.

在本发明的一种优选实施方式中，所述立体视觉的方法为：In a preferred embodiment of the present invention, the method for stereo vision is:

S31：根据主摄像头A和辅助摄像头B的安装位置，分别进行标定，得出摄像头的内参数、外参数，以及主摄像头A和辅助摄像头B之间的间距b；S31: According to the installation positions of the main camera A and the auxiliary camera B, perform calibration respectively to obtain the internal parameters and external parameters of the cameras, and the distance b between the main camera A and the auxiliary camera B;

S32：对辅助摄像头B获取的图像采用基于单幅图像的障碍物检测方法，获取障碍物中心位置、上边界中心位置在图像坐标系下的二维坐标；S32: Using an obstacle detection method based on a single image for the image acquired by the auxiliary camera B, acquire the two-dimensional coordinates of the center position of the obstacle and the center position of the upper boundary in the image coordinate system;

S33：利用如下公式获取障碍物在主摄像头坐标系下的三维坐标x_c,y_c,z_c，并利用摄像头的标定参数，得出障碍物在世界坐标系下的三维坐标，S33: Use the following formula to obtain the three-dimensional coordinates x_c , y_c , z_c of the obstacle in the main camera coordinate system, and use the calibration parameters of the camera to obtain the three-dimensional coordinates of the obstacle in the world coordinate system,

$<span><span>(</span>(</span>{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>c</span>c</span>}}<span><span>,</span>,</span>{\mathrm{<span><span>y</span>the\; y</span>}}_{\mathrm{<span><span>c</span>c</span>}}<span><span>,</span>,</span>{\mathrm{<span><span>z</span>z</span>}}_{\mathrm{<span><span>c</span>c</span>}}<span><span>)</span>)</span><span><span>=</span>=</span><span><span>(</span>(</span>\frac{{\mathrm{<span><span>bu</span>bu</span>}}_{\mathrm{<span><span>1</span>1</span>}}}{{\mathrm{<span><span>u</span>u</span>}}_{\mathrm{<span><span>1</span>1</span>}}<span><span>-</span>-</span>{\mathrm{<span><span>u</span>u</span>}}_{\mathrm{<span><span>2</span>2</span>}}}<span><span>,</span>,</span>\frac{{\mathrm{<span><span>bv</span>bv</span>}}_{\mathrm{<span><span>1</span>1</span>}}}{{\mathrm{<span><span>u</span>u</span>}}_{\mathrm{<span><span>1</span>1</span>}}<span><span>-</span>-</span>{\mathrm{<span><span>u</span>u</span>}}_{\mathrm{<span><span>2</span>2</span>}}}<span><span>,</span>,</span>\frac{\mathrm{<span><span>bf</span>b\; f</span>}}{{\mathrm{<span><span>u</span>u</span>}}_{\mathrm{<span><span>1</span>1</span>}}<span><span>-</span>-</span>{\mathrm{<span><span>u</span>u</span>}}_{\mathrm{<span><span>2</span>2</span>}}}<span><span>)</span>)</span><span><span>,</span>,</span>$

其中，b为两摄像头的间距，f为摄像头的焦距（u₁，v₁）为标定点在A摄像头平面的投影位置，（u₂，v₂）为标定点在B摄像头平面的投影位置，并且v₁=v₂。Among them, b is the distance between the two cameras, f is the focal length of the camera (u₁ , v₁ ) is the projection position of the calibration point on the plane of camera A, (u₂ , v₂ ) is the projection position of the calibration point on the plane of camera B, And v₁ =v₂ .

在本发明的另一种优选实施方式中，所述分段式自适应控制策略是指：根据车身距离车道中间的偏移角度和距离，以及车身此刻的行驶速度，根据参数degree_per_dist自适应适度调整下一时刻的车身转角和车速，当车辆行驶在直道的中间附近位置时，始终保持车身以全速直行的方向行驶，若偏向车道两侧的任意一侧，则进行微调。In another preferred embodiment of the present invention, the segmented adaptive control strategy refers to: according to the offset angle and distance of the vehicle body from the middle of the lane, and the current driving speed of the vehicle body, the parameter degree_per_dist is adaptively and moderately adjusted The body corner and speed at the next moment, when the vehicle is driving near the middle of the straight road, always keep the body running in the direction of full speed straight, if it deviates to either side of the lane, fine-tune it.

为了实现本发明的上述目的，根据本发明的另一个方面，本发明提供了一种基于机器视觉的车辆避障系统，包括主摄像头A、辅助摄像头B、测距装置、处理器和控制器，所述主摄像头A和辅助摄像头B分别用于获取路面的图像并将所述图像传输给所述处理器；所述处理器用于判定缩微车前方是否存在障碍物并将障碍物距离和障碍物所处车道的检测信息传输给控制器，同时处理器对主摄像头A和辅助摄像头B进行标定，判定障碍物中心位置、上边界中心位置的三维坐标，从而确定障碍物的高度并将所述障碍物高度信息传输给控制器；所述测距装置位于车身两侧，用于检测相邻两车道的车辆状况并将可行使区域的信息传输给所述控制器；控制器与运行控制模块相连，所述运行控制模块与车轮相连，控制器根据获得的障碍物距离、障碍物所处车道、障碍物高度以及相邻两车道的可行使区域信息，向运行控制模块发送命令，所述运行控制模块控制车轮的转向和速度，完成车辆的自主换道。In order to achieve the above object of the present invention, according to another aspect of the present invention, the present invention provides a vehicle obstacle avoidance system based on machine vision, comprising a main camera A, an auxiliary camera B, a distance measuring device, a processor and a controller, The main camera A and the auxiliary camera B are respectively used to acquire images of the road surface and transmit the images to the processor; At the same time, the processor calibrates the main camera A and the auxiliary camera B, and determines the three-dimensional coordinates of the center position of the obstacle and the center position of the upper boundary, so as to determine the height of the obstacle and place the obstacle The height information is transmitted to the controller; the distance measuring device is located on both sides of the vehicle body, and is used to detect the vehicle condition of the adjacent two lanes and transmit the information of the exercisable area to the controller; the controller is connected with the operation control module, and the The above operation control module is connected with the wheels, and the controller sends commands to the operation control module according to the obtained obstacle distance, the lane where the obstacle is located, the obstacle height, and the information on the feasible areas of the two adjacent lanes, and the operation control module controls The steering and speed of the wheels complete the autonomous lane change of the vehicle.

本发明的有益效果是：The beneficial effects of the present invention are:

本发明首先采用基于单幅图像的障碍物检测算法判定前方是否存在障碍物，避免了直接采用立体视觉系统进行检测导致的实时性不够的问题，检测周期短；然后进一步利用双摄像头，运用立体视觉系统判定障碍物的高度信息；本发明的避障方法稳定、自适应程度高，车辆避障姿态流畅，避障成功率高达98%以上。The present invention first adopts an obstacle detection algorithm based on a single image to determine whether there is an obstacle ahead, avoiding the problem of insufficient real-time performance caused by directly using a stereo vision system for detection, and the detection cycle is short; then further using dual cameras, using stereo vision The system determines the height information of the obstacle; the obstacle avoidance method of the present invention is stable, has a high degree of self-adaptation, the vehicle obstacle avoidance posture is smooth, and the obstacle avoidance success rate is as high as 98%.

本发明采用立体视觉的方法障碍物的高度信息，兼顾了系统的实时性和准确性要求。The present invention adopts the height information of the obstacle by the method of stereo vision, taking into account the real-time and accuracy requirements of the system.

本发明根据车辆所处的状态模式，采用分段式自适应控制策略，对底层控制器发送控制命令。根据车身距离车道中间的偏移角度和距离，以及车身此刻的行驶速度，根据参数degree_per_dist自适应适度调整下一时刻的车身转角和车速。直道作为弯道的特殊道路类型，当车辆行驶在直道的中间附近位置时，始终保持车身以全速直行的方向行驶，若偏向车道两侧的任意一侧，则根据分段式自适应控制策略进行适度微调。相比较其他控制方法，本发明中方法可有效减少车身舵机角度的调整次数，摆脱车身因频繁调整行驶方向导致的车身左右摇晃，从而保障车身相对稳定的行驶。According to the state mode of the vehicle, the present invention adopts a segmented self-adaptive control strategy to send control commands to the bottom controller. According to the offset angle and distance of the vehicle body from the middle of the lane, and the driving speed of the vehicle body at the moment, according to the parameter degree_per_dist, the body corner and vehicle speed at the next moment are adaptively adjusted. The straight road is a special type of road with curves. When the vehicle is driving near the middle of the straight road, it will always keep the vehicle body running straight at full speed. Moderate fine-tuning. Compared with other control methods, the method of the present invention can effectively reduce the number of adjustments of the angle of the steering gear of the vehicle body, and get rid of the side-to-side shaking of the vehicle body caused by frequent adjustments of the driving direction, thereby ensuring relatively stable driving of the vehicle body.

本发明分段式自适应控制策略可有效减少车身舵机角度的调整次数，摆脱车身因频繁调整行驶方向导致的车身左右摇晃，从而保障车身相对稳定的行驶。The segmented self-adaptive control strategy of the present invention can effectively reduce the number of times of adjusting the angle of the steering gear of the vehicle body, and get rid of the left and right shaking of the vehicle body caused by frequent adjustment of the driving direction of the vehicle body, thereby ensuring relatively stable driving of the vehicle body.

本发明的附加方面和优点将在下面的描述中部分给出，部分将从下面的描述中变得明显，或通过本发明的实践了解到。Additional aspects and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.

附图说明Description of drawings

本发明的上述和/或附加的方面和优点从结合下面附图对实施例的描述中将变得明显和容易理解，其中：The above and/or additional aspects and advantages of the present invention will become apparent and comprehensible from the description of the embodiments in conjunction with the following drawings, wherein:

图1是本发明一种优选实施方式中缩微智能车硬件架构；Fig. 1 is a miniature smart car hardware architecture in a preferred embodiment of the present invention;

图2是本发明基于机器视觉的车辆避障方法流程图。Fig. 2 is a flow chart of the vehicle obstacle avoidance method based on machine vision in the present invention.

具体实施方式Detailed ways

下面详细描述本发明的实施例，所述实施例的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的，仅用于解释本发明，而不能理解为对本发明的限制。Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals designate the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary only for explaining the present invention and should not be construed as limiting the present invention.

在本发明的描述中，除非另有规定和限定，需要说明的是，术语“安装”、“相连”、“连接”应做广义理解，例如，可以是机械连接或电连接，也可以是两个元件内部的连通，可以是直接相连，也可以通过中间媒介间接相连，对于本领域的普通技术人员而言，可以根据具体情况理解上述术语的具体含义。In the description of the present invention, unless otherwise specified and limited, it should be noted that the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be mechanical connection or electrical connection, or two The internal communication of each element may be directly connected or indirectly connected through an intermediary. Those skilled in the art can understand the specific meanings of the above terms according to specific situations.

本发明可以应用于实际的车辆，也可以应用于缩微车，在本发明的一种优选实施方式中，采用缩微车进行说明，具体采用的缩微车与真车按照1:10的缩微比获得。The present invention can be applied to actual vehicles, and can also be applied to miniature cars. In a preferred embodiment of the present invention, a miniature car is used for illustration, and the miniature car and the real car are obtained according to the miniature ratio of 1:10.

本发明提供了一种基于机器视觉的缩微车，其包括底盘，在该底盘上设置有马达和舵机。该缩微车还包括摄像头、主板和控制板，摄像头用于获取路面图片，路面图片用于识别车道线、道路标识及障碍物等信息；主板的处理器与摄像头相连，主板根据摄像头传输的信息运行控制程序并下达控制命令，控制板与主板相连，控制板接收主板下达的控制命令通过运行控制模块控制马达和舵机的运行，控制车轮的行进。The invention provides a miniature car based on machine vision, which includes a chassis on which a motor and a steering gear are arranged. The miniature car also includes a camera, a main board and a control board. The camera is used to obtain road pictures, and the road pictures are used to identify information such as lane lines, road signs and obstacles; the processor of the main board is connected to the camera, and the main board operates according to the information transmitted by the camera. Control the program and issue control commands. The control board is connected to the main board. The control board receives the control commands issued by the main board and controls the operation of the motor and steering gear through the operation control module to control the advancement of the wheels.

本发明还提供了一种基于机器视觉的缩微智能车避障系统，如图1所示，包括主摄像头A、辅助摄像头B、测距装置、处理器和控制器，所述主摄像头A和辅助摄像头B分别用于获取路面的图像并将所述图像传输给所述处理器；所述处理器用于判定缩微车前方是否存在障碍物并将障碍物距离和障碍物所处车道的检测信息传输给控制器，同时处理器对主摄像头A和辅助摄像头B进行标定，判定障碍物中心位置、上边界中心位置的三维坐标，从而确定障碍物的高度并将所述障碍物高度信息传输给控制器；所述测距装置位于车身两侧，用于检测相邻两车道的车辆状况并将可行使区域的信息传输给所述控制器；控制器与运行控制模块相连，所述运行控制模块与车轮相连，控制器根据获得的障碍物距离、障碍物所处车道、障碍物高度以及相邻两车道的可行使区域信息，向运行控制模块发送命令，所述运行控制模块控制车轮的转向和速度，完成车辆的自主换道。The present invention also provides a miniature smart car obstacle avoidance system based on machine vision, as shown in Figure 1, comprising a main camera A, an auxiliary camera B, a distance measuring device, a processor and a controller, the main camera A and the auxiliary The camera B is used to acquire the image of the road surface and transmits the image to the processor; the processor is used to determine whether there is an obstacle in front of the miniature car and transmit the detection information of the obstacle distance and the lane where the obstacle is located to a controller, and the processor calibrates the main camera A and the auxiliary camera B at the same time, and determines the three-dimensional coordinates of the center position of the obstacle and the center position of the upper boundary, thereby determining the height of the obstacle and transmitting the height information of the obstacle to the controller; The distance measuring device is located on both sides of the vehicle body, and is used to detect the vehicle conditions of the adjacent two lanes and transmit the information of the driving area to the controller; the controller is connected to the operation control module, and the operation control module is connected to the wheels , the controller sends commands to the operation control module according to the obtained obstacle distance, the lane where the obstacle is located, the height of the obstacle, and the information on the feasible areas of the two adjacent lanes. The operation control module controls the steering and speed of the wheels to complete Autonomous lane changing of vehicles.

在本实施方式中，该缩微车底盘选用HPI cup racer底盘，该底盘带有电机和舵机，用于机械运动。控制板采用DFR0003型号的Arduino控制板。主板采用嵌入式x86主板。摄像头可以为1个，也可以为多个，当摄像头为多个时，每一个摄像头均与主板相连，在本发明的一个优选实施方中，摄像头为两个，在本发明的一个更加优选实施方中，摄像头为罗技摄像头。In this embodiment, the miniature car chassis selects an HPI cup racer chassis, which has a motor and a steering gear for mechanical movement. The control board adopts the Arduino control board of the DFR0003 model. The motherboard uses an embedded x86 motherboard. Camera can be 1, also can be a plurality of, when camera is multiple, each camera is all connected with mainboard, in a preferred embodiment of the present invention, camera is two, in a more preferred implementation of the present invention In the square, the camera is a Logitech camera.

本发明的基于机器视觉的缩微车通过主板和控制板根据视觉传感器获取的路面图片信息控制马达和舵机的运行，该缩微车相对于原尺度车辆，缩微智能车的结构简单，造价低廉，多车测试环境容易构建。并且实验场地和环境容易调整，可以方便地进行多种不同环境下的实验。The miniature car based on machine vision of the present invention controls the operation of the motor and steering gear according to the road surface picture information obtained by the visual sensor through the main board and the control board. Compared with the original scale vehicle, the miniature car has a simple structure, low cost, and many The vehicle test environment is easy to build. And the experimental site and environment are easy to adjust, and experiments in various environments can be conveniently carried out.

本发明缩微车的硬件架构包括设计合理的与车身长度、空间结构相匹配的整车硬件架构，在本实施方式中，将缩微车的车身空间分为上、中、下三层结构，在每层结构中放置对应的硬件装置。控制板、底盘等硬件位于车身下层，在中层放置主板，在上层车身空间设置视觉传感器。底层控制器和处理器分别供电。图像摄取装置安装于智能车的上方距离地面20-25cm处。The hardware architecture of the miniature car of the present invention includes a well-designed whole vehicle hardware architecture that matches the body length and spatial structure. In this embodiment, the body space of the miniature car is divided into three layers: upper, middle and lower. The corresponding hardware devices are placed in the layer structure. The control panel, chassis and other hardware are located in the lower layer of the body, the main board is placed in the middle layer, and the visual sensor is installed in the upper body space. The underlying controller and processor are powered separately. The image capture device is installed above the smart car at a distance of 20-25cm from the ground.

在本发明的一种优选实施方式中，该缩微车还包括用于判断运行方向及上下坡角度的电子罗盘和用于判断前后车距及相邻车道缩微车距离的红外测距传感器，电子罗盘和红外测距传感器分别与控制板相连，控制板根据电子罗盘和红外测距传感器传输的信息控制马达和舵机的运行。In a preferred embodiment of the present invention, the miniature car also includes an electronic compass for judging the running direction and uphill and downhill angles and an infrared ranging sensor for judging the distance between the front and rear cars and the distance between the miniature cars in adjacent lanes. and the infrared distance measuring sensor are respectively connected with the control board, and the control board controls the operation of the motor and the steering gear according to the information transmitted by the electronic compass and the infrared distance measuring sensor.

在本发明的一个优选实施方式中，红外测距传感器为两路，一路用于判断前后车距，另一路用于判断该缩微车与相邻车道的缩微车的距离。In a preferred embodiment of the present invention, there are two infrared ranging sensors, one for judging the distance between front and rear vehicles, and the other for judging the distance between the miniature car and the miniature cars in the adjacent lane.

本发明通过利用电子罗盘和红外测距传感器，实现了对缩微车辆行驶的精确控制，大大提高了行驶的安全性。By using the electronic compass and the infrared distance measuring sensor, the invention realizes the precise control of the running of the miniature vehicle and greatly improves the driving safety.

在本实施方式中，该缩微车还包括电源，本发明的一种优选实施方式中，采用12V锂电池给主板供电；采用8V锂电池给电机供电；视觉传感器和控制板均由主板供电。In this embodiment, the miniature car also includes a power supply. In a preferred embodiment of the present invention, a 12V lithium battery is used to supply power to the main board; an 8V lithium battery is used to supply power to the motor; both the visual sensor and the control board are powered by the main board.

在本实施方式中，缩微车还包括车壳，为了便于实现对缩微车的研究，需要考虑各配件的安置情况，包括重量、尺寸因素，由于X86主板和马达耗电量高，若选用大功率电池会增加缩微车重量，因此需要选用尺寸小，功率合适的电池，同时车壳的选用也需要考虑尺寸因素。在本发明的一种优选实施方式中，外形设计时遵循以下原则：In this embodiment, the miniature car also includes a car shell. In order to facilitate the research on the miniature car, it is necessary to consider the placement of each accessory, including weight and size factors. Due to the high power consumption of the X86 motherboard and motor, if a high-power The battery will increase the weight of the miniature car, so it is necessary to choose a battery with a small size and a suitable power. At the same time, the selection of the car shell also needs to consider the size factor. In a preferred embodiment of the present invention, the following principles are followed when designing the shape:

1、主板电池和马达电池能够方便的安装、拆卸，以方便充电；1. The motherboard battery and motor battery can be easily installed and disassembled to facilitate charging;

2、控制板电池无需经常拆卸，但须预留充电口；2. The control board battery does not need to be disassembled frequently, but a charging port must be reserved;

3、车壳必须方便拆卸，换装电池。可以根据车身的大小定制车壳，考虑电池和车壳统一设计，方便更换电池。3. The car shell must be easy to disassemble and replace the battery. The car shell can be customized according to the size of the car body, and the unified design of the battery and car shell is considered to facilitate battery replacement.

本发明还提供了一种基于机器视觉的缩微智能车避障方法，如图2所示，其包括如下步骤：The present invention also provides a kind of miniature smart car obstacle avoidance method based on machine vision, as shown in Figure 2, it comprises the following steps:

S1：处理器采用基于单幅图像的障碍物检测算法判定缩微车前方是否存在障碍物，如果有障碍物，则将障碍物距离和障碍物所处车道的检测信息传输给控制器，并执行步骤S2，如果没有障碍物，则继续执行步骤S1；S1: The processor uses an obstacle detection algorithm based on a single image to determine whether there is an obstacle in front of the miniature car. If there is an obstacle, it transmits the detection information of the obstacle distance and the lane where the obstacle is located to the controller, and executes the steps S2, if there is no obstacle, proceed to step S1;

S2：处理器对两个摄像头进行标定，采用立体视觉的方法判定障碍物中心位置、上边界中心位置的三维坐标，从而确定障碍物的高度并将所述障碍物高度信息传输给控制器；S2: The processor calibrates the two cameras, and uses stereo vision to determine the three-dimensional coordinates of the center position of the obstacle and the center position of the upper boundary, so as to determine the height of the obstacle and transmit the height information of the obstacle to the controller;

S3：车身两侧的测距装置检测相邻两车道的车辆状况，为避障换道提供可行驶的区域并将所述可行使区域的信息传输给所述控制器；S3: The distance measuring devices on both sides of the vehicle body detect the vehicle conditions of the adjacent two lanes, provide a drivable area for obstacle avoidance and lane change, and transmit the information of the drivable area to the controller;

S4：控制器根据获得的障碍物距离、障碍物所处车道、障碍物高度以及相邻两车道的可行使区域信息，采用自适应换道策略，向运行控制模块发送命令，控制车轮的转向和速度，完成车辆的自主换道。S4: The controller adopts an adaptive lane-changing strategy based on the obtained obstacle distance, the lane where the obstacle is located, the height of the obstacle, and the maneuverable area information of the two adjacent lanes, and sends commands to the operation control module to control the steering and steering of the wheels. speed to complete the autonomous lane change of the vehicle.

在本实施方式中，所述单幅图像的障碍物检测算法的步骤为：In this embodiment, the steps of the obstacle detection algorithm of the single image are:

S21：对图像进行色彩空间变换RGB转HSV，并单独抽取S通道图像；S21: Perform color space transformation on the image from RGB to HSV, and separately extract the S channel image;

S22：对S图像利用大津法进行动态阈值二值化得到二值图像Bin，对Bin图像去噪，消除干扰；S22: Perform dynamic threshold binarization on the S image using the Otsu method to obtain a binary image Bin, denoise the Bin image, and eliminate interference;

S23：在Bin图像上搜索所有可能存在的障碍物轮廓，根据设置的筛选条件将误判的障碍物剔除；S23: Search for all possible obstacle contours on the Bin image, and remove misjudged obstacles according to the set filter conditions;

S24：并根据车道线的方位判定障碍物所处车道。S24: and determine the lane where the obstacle is located according to the orientation of the lane line.

在本实施方式中，所述立体视觉的方法为：In this embodiment, the stereo vision method is:

S31：根据主摄像头A和辅助摄像头B的安装位置，分别进行标定，得出摄像头的内参数、外参数，以及主摄像头A和辅助摄像头B之间的间距b，在本实施方式中，摄像头的内参数、外参数可以按照现有技术中的获取方法得出；S31: According to the installation positions of the main camera A and the auxiliary camera B, perform calibration respectively to obtain the internal parameters and external parameters of the camera, and the distance b between the main camera A and the auxiliary camera B. In this embodiment, the camera's The internal parameter and the external parameter can be obtained according to the acquisition method in the prior art;

S32：对辅助摄像头B获取的图像采用基于单幅图像的障碍物检测方法，获取障碍物中心位置、上边界中心位置在图像坐标系下的二维坐标；S32: Using an obstacle detection method based on a single image for the image acquired by the auxiliary camera B, acquire the two-dimensional coordinates of the center position of the obstacle and the center position of the upper boundary in the image coordinate system;

S33：利用如下公式获取障碍物在主摄像头坐标系下的三维坐标x_c,y_c,z_c，并利用摄像头的标定的内参数和外参数，得出障碍物在世界坐标系下的三维坐标，S33: Use the following formula to obtain the three-dimensional coordinates x_c , y_c , z_c of the obstacle in the main camera coordinate system, and use the calibrated internal parameters and external parameters of the camera to obtain the three-dimensional coordinates of the obstacle in the world coordinate system ,

$<span><span>(</span>(</span>{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>c</span>c</span>}}<span><span>,</span>,</span>{\mathrm{<span><span>y</span>the\; y</span>}}_{\mathrm{<span><span>c</span>c</span>}}<span><span>,</span>,</span>{\mathrm{<span><span>z</span>z</span>}}_{\mathrm{<span><span>c</span>c</span>}}<span><span>)</span>)</span><span><span>=</span>=</span><span><span>(</span>(</span>\frac{{\mathrm{<span><span>bu</span>bu</span>}}_{\mathrm{<span><span>1</span>1</span>}}}{{\mathrm{<span><span>u</span>u</span>}}_{\mathrm{<span><span>1</span>1</span>}}<span><span>-</span>-</span>{\mathrm{<span><span>u</span>u</span>}}_{\mathrm{<span><span>2</span>2</span>}}}<span><span>,</span>,</span>\frac{{\mathrm{<span><span>bv</span>bv</span>}}_{\mathrm{<span><span>1</span>1</span>}}}{{\mathrm{<span><span>u</span>u</span>}}_{\mathrm{<span><span>1</span>1</span>}}<span><span>-</span>-</span>{\mathrm{<span><span>u</span>u</span>}}_{\mathrm{<span><span>2</span>2</span>}}}<span><span>,</span>,</span>\frac{\mathrm{<span><span>bf</span>b\; f</span>}}{{\mathrm{<span><span>u</span>u</span>}}_{\mathrm{<span><span>1</span>1</span>}}<span><span>-</span>-</span>{\mathrm{<span><span>u</span>u</span>}}_{\mathrm{<span><span>2</span>2</span>}}}<span><span>)</span>)</span><span><span>,</span>,</span>$

其中，b为两摄像头的间距，f为摄像头的焦距，，（u₁，v₁）为标定点在A摄像头平面的投影位置，（u₂，v₂）为标定点在B摄像头平面的投影位置，在本实施方式中，标定点在A摄像头平面的投影位置与在B摄像头平面的投影位置的纵坐标相等，即有v₁=v₂；Among them, b is the distance between the two cameras, f is the focal length of the camera, (u₁ , v₁ ) is the projection position of the calibration point on the A camera plane, (u₂ , v₂ ) is the projection of the calibration point on the B camera plane Position, in this embodiment, the projection position of the calibration point on the A camera plane is equal to the ordinate of the projection position on the B camera plane, that is, v₁ =v₂ ;

在本实施方式中，所述分段式自适应控制策略是指：根据车身距离车道中间的偏移角度和距离，以及车身此刻的行驶速度，根据参数degree_per_dist自适应适度调整下一时刻的车身转角和车速，当车辆行驶在直道的中间附近位置时，始终保持车身以全速直行的方向行驶，若偏向车道两侧的任意一侧，则进行微调。In this embodiment, the segmented adaptive control strategy refers to: according to the offset angle and distance of the vehicle body from the middle of the lane, and the current driving speed of the vehicle body, according to the parameter degree_per_dist, adaptively adjust the vehicle body rotation angle at the next moment and vehicle speed, when the vehicle is driving near the middle of the straight road, always keep the vehicle body running straight at full speed, if it deviates to either side of the lane, fine-tune it.

本发明在步骤S1之前还具有以下步骤：通过巡线系统进行巡线，确定车道线。在本实施方式中，缩微车的巡线系统包括图像获取模块、主板和控制器，所述图像获取模块用于获取视觉传感器采集的路面的RGB彩色图像并将所述RGB彩色图像传输给所述主板，所述RGB彩色图像中含有车道线信息；所述主板包括巡线模块，所述巡线模块包括图像处理模块、二值化模块、边缘检测模块和处理器；所述图像处理模块接收所述图像获取模块获取的RGB彩色图像并将所述RGB彩色图像转化为灰度图像；所述二值化模块与所述图像处理模块相连，用于接收所述灰度图像并获取所述灰度图像中每帧图像的最佳动态阈值并进行图像分割，得到二值图像，将车道线分离出来；所述边缘检测模块与所述二值化模块相连，用于接收并对所述二值化图像进行边缘检测，得到含有车道线的内、外边缘的边缘图像；所述处理器与所述边缘检测模块相连，所述处理器利用霍夫变换在所述边缘图像中检测车道线并获取车道线参数，建立车道线模型；处理器利用获取的所述车道线参数，通过逆透视变换获取车辆在世界坐标系中所处的车道位置数据，以及车辆转角和距离参数，判别车辆的行驶模式；同时利用获取的车辆转角以及距离参数，根据车辆的行驶模式，采用分段式自适应控制策略，向控制器发送控制命令；所述控制器分别与处理器、舵机和电机相连，用于接收所述处理器的控制命令，并根据控制命令调整舵机和电机的工作参数，实时控制车辆的行车方向以及行车速度。The present invention also has the following steps before step S1: performing line inspection through a line inspection system to determine lane lines. In this embodiment, the line inspection system of the miniature car includes an image acquisition module, a main board and a controller, and the image acquisition module is used to acquire the RGB color image of the road surface collected by the visual sensor and transmit the RGB color image to the Mainboard, the RGB color image contains lane line information; the mainboard includes a line inspection module, and the line inspection module includes an image processing module, a binarization module, an edge detection module and a processor; the image processing module receives all The RGB color image acquired by the image acquisition module and convert the RGB color image into a grayscale image; the binarization module is connected with the image processing module for receiving the grayscale image and obtaining the grayscale The optimal dynamic threshold of each frame of image in the image and image segmentation to obtain a binary image and separate the lane lines; the edge detection module is connected to the binarization module for receiving and performing the binarization The image is edge detected to obtain an edge image containing the inner and outer edges of the lane line; the processor is connected to the edge detection module, and the processor uses Hough transform to detect the lane line in the edge image and obtain the lane line parameters to establish a lane line model; the processor uses the obtained lane line parameters to obtain the lane position data of the vehicle in the world coordinate system through inverse perspective transformation, as well as the vehicle corner and distance parameters to determine the driving mode of the vehicle; At the same time, using the obtained vehicle angle and distance parameters, according to the driving mode of the vehicle, a segmented adaptive control strategy is adopted to send control commands to the controller; the controller is respectively connected to the processor, steering gear and motor for receiving The processor controls the command, adjusts the working parameters of the steering gear and the motor according to the control command, and controls the driving direction and driving speed of the vehicle in real time.

在本实施方式中，主板还包括初始化模块、障碍物检测模块、红绿灯检测模块、交通标志检测模块、地面标志检测模块；所述初始化模块分别与所述视觉传感器、所述马达和所述舵机相连，用于对所述视觉传感器、所述马达和所述舵机进行初始化；所述障碍物检测模块与所述视觉传感器相连，用于根据所述视觉传感器获取的路面图片检测车辆前方的障碍物；所述红绿灯检测模块与所述视觉传感器相连，用于根据所述视觉传感器获取的路面图片检测红绿灯的工作情况；所述交通标志检测模块与所述视觉传感器相连，用于根据所述视觉传感器获取的路面图片对交通标志进行识别和判断；所述地面标志检测模块与所述视觉传感器相连，用于根据所述视觉传感器获取的路面图片识别车辆所处车道的地面标志。本发明通过主板的初始化模块对视觉传感器、马达和舵机进行初始化，提高了行驶的准确性，另外，该主板能够具有的寻线模块、障碍物检测模块、红绿灯检测模块、交通标志检测模块、地面标志检测模块，实现了自动车道线跟踪、道路位置检测、自动换道、多车互动和分析车辆目标方向的能力，提高了行驶的安全性。In this embodiment, the mainboard also includes an initialization module, an obstacle detection module, a traffic light detection module, a traffic sign detection module, and a ground sign detection module; the initialization module is connected to the visual sensor, the motor and the steering gear respectively connected to the visual sensor, the motor and the steering gear; the obstacle detection module is connected to the visual sensor and used to detect obstacles in front of the vehicle according to the road surface pictures acquired by the visual sensor The traffic light detection module is connected with the visual sensor, and is used to detect the working condition of the traffic light according to the road surface picture obtained by the visual sensor; the traffic sign detection module is connected with the visual sensor, and is used for detecting the traffic light according to the visual sensor. The road surface picture acquired by the sensor identifies and judges the traffic sign; the ground sign detection module is connected to the visual sensor, and is used to identify the ground sign of the lane where the vehicle is located according to the road surface picture acquired by the visual sensor. The present invention initializes the visual sensor, motor and steering gear through the initialization module of the main board, which improves the accuracy of driving. In addition, the main board can have a line hunting module, an obstacle detection module, a traffic light detection module, a traffic sign detection module, The ground mark detection module realizes automatic lane line tracking, road position detection, automatic lane change, multi-vehicle interaction and the ability to analyze the direction of vehicle targets, improving driving safety.

在本发明的另一种优选实施方式中，主板还包括速度控制模块和方向控制模块，速度控制模块和所述方向控制模块两者分别与初始化模块、寻线模块、障碍物检测模块、红绿灯检测模块、交通标志检测模块和地面标志检测模块相连。速度控制模块接收所述初始化模块、寻线模块、障碍物检测模块、红绿灯检测模块、交通标志检测模块、地面标志检测模块检测的信息并产生速度控制信息，所述速度控制模块将速度控制信息传输给所述控制板，所述控制板控制所述马达的运行速度。方向控制模块接收所述初始化模块、寻线模块、障碍物检测模块、红绿灯检测模块、交通标志检测模块、地面标志检测模块检测的信息，并产生方向控制信息，所述方向控制模块将方向控制信息传输给所述控制板，所述控制板控制舵机的方向。本发明的主板通过速度控制模块和方向控制模块控制马达的运行速度和舵机的方向，实现缩微车在对周围环境的感知后应采取的行为决策，包括寻线行走，壁障，遵守交通规则等，提高了行驶的安全性。In another preferred embodiment of the present invention, the mainboard also includes a speed control module and a direction control module, and the speed control module and the direction control module are respectively connected with the initialization module, the line hunting module, the obstacle detection module, and the traffic light detection module. The module, the traffic sign detection module and the ground sign detection module are connected. The speed control module receives the information detected by the initialization module, the line hunting module, the obstacle detection module, the traffic light detection module, the traffic sign detection module, and the ground sign detection module and generates speed control information, and the speed control module transmits the speed control information To the control board, the control board controls the operating speed of the motor. The direction control module receives the information detected by the initialization module, the line hunting module, the obstacle detection module, the traffic light detection module, the traffic sign detection module, and the ground sign detection module, and generates direction control information, and the direction control module converts the direction control information transmitted to the control board, and the control board controls the direction of the steering gear. The main board of the present invention controls the running speed of the motor and the direction of the steering gear through the speed control module and the direction control module, so as to realize the behavior decisions that the miniature car should take after sensing the surrounding environment, including line hunting, barriers, and compliance with traffic rules etc., improving the safety of driving.

在本发明的一种优选实施方式中，主板采用嵌入式x86主板，基于该主板的缩微车具有自动车道线跟踪、道路位置检测、自动换道、多车互动和分析车辆目标方向的能力。在本实施方式中，主板具有USB接口，其数量可以根据需要连接的硬件进行调整，可以为限不限于4路，其中，2路USB接口连接摄像头，1路USB接口连接无线网卡；1路USB接口连接控制板。控制板上有多路数字I/O口，用于连接红外测距传感器、电子罗盘，马达和舵机等设备。In a preferred embodiment of the present invention, the motherboard adopts an embedded x86 motherboard, and the miniature car based on the motherboard has the ability of automatic lane line tracking, road position detection, automatic lane change, multi-vehicle interaction and analysis of vehicle target direction. In this embodiment, the motherboard has USB interfaces, the number of which can be adjusted according to the hardware that needs to be connected, and can be limited to but not limited to 4 channels, wherein, 2 USB interfaces are connected to the camera, and 1 USB interface is connected to the wireless network card; The interface is connected to the control board. There are multiple digital I/O ports on the control board, which are used to connect infrared ranging sensors, electronic compass, motors, steering gears and other equipment.

本发明主板的各模块能够实现复杂道路环境下动、静态目标环境感知的相关算法，包括寻线算法、障碍物检测算法、红绿灯检测算法、交通标志检测算法、地面标志检测算法；实现缩微车在对周围环境的感知后应采取的行为决策，包括寻线行走，壁障，遵守交通规则等。Each module of the mainboard of the present invention can realize related algorithms of dynamic and static target environment perception in complex road environments, including line-finding algorithms, obstacle detection algorithms, traffic light detection algorithms, traffic sign detection algorithms, and ground sign detection algorithms; Behavior decisions should be taken after the perception of the surrounding environment, including line hunting, barriers, obeying traffic rules, etc.

本发明还提供了一种基于机器视觉的缩微车控制系统，其包括至少一台缩微车、服务器和管理端电脑，服务器与缩微车的主板相连，服务器与主板进行双向信息交互；管理端电脑与服务器相连，管理端电脑接收服务器传输的信息并通过服务器对所述缩微车的运行进行监控管理。The present invention also provides a miniature car control system based on machine vision, which includes at least one miniature car, a server and a management computer, the server is connected to the main board of the miniature car, and the server and the main board perform two-way information interaction; the management computer and the main board The server is connected, and the management terminal computer receives the information transmitted by the server and monitors and manages the operation of the miniature car through the server.

在本实施方式中，缩微车与服务器通过TCP连接，也可以通过无线网络将缩微车终端与服务器连接起来，服务器将信息传输给管理端电脑，管理端电脑实时监控缩微车的运行状态，实现对缩微车的监控管理。In this embodiment, the miniature car is connected to the server through TCP, and the miniature car terminal can also be connected to the server through a wireless network. Monitoring and management of microcars.

本发明的基于机器视觉的缩微车控制系统的搭建是基于PC平台基础上实现的，通过视觉传感器在模拟的高仿真缩微车测试平台上对缩微车的环境感知与智能行为决策能力进行测试。缩微车是通过无线通信等技术手段将获取的信息连接到服务器的网络中,然后加以分析，得出策略，从而实现对缩微车行为的控制，从而达到车与路、车与车、车与城市网络实现相互连接。在该缩微车控制系统上，由于缩微车的实验环境相对封闭，而且缩微车基本不存在安全方面的问题，因此自主驾驶实验不会受到法律法规等非技术方面的约束。另外相对于原尺度车辆，缩微车的结构简单，造价低廉，多车测试环境容易构建。并且实验场地和环境容易调整，可以方便地进行多种不同环境下的实验。缩微车的研究是伴随着城市交通拥堵的日益加重以及智能交通解决方案技术的不断进步而出现的，也是城市智能交通以热点区域为主、以车为对象的管理模式转变的重要体现之一，推动了我国智能交通的向前发展。The machine vision-based miniature car control system of the present invention is built on the basis of a PC platform, and the environmental perception and intelligent behavior decision-making ability of the miniature car are tested on the simulated high-simulation miniature car test platform through visual sensors. The miniature car is to connect the obtained information to the server network through wireless communication and other technical means, and then analyze it to obtain a strategy, so as to realize the control of the behavior of the miniature car, so as to achieve the goal of car and road, car and car, car and city. Networks are interconnected. In the miniature car control system, since the experimental environment of the miniature car is relatively closed, and there are basically no safety issues in the miniature car, the autonomous driving experiment will not be subject to non-technical constraints such as laws and regulations. In addition, compared with the original scale vehicle, the structure of the miniature vehicle is simple, the cost is low, and the multi-vehicle test environment is easy to build. And the experimental site and environment are easy to adjust, and experiments in various environments can be conveniently carried out. The research on microcars emerged along with the aggravation of urban traffic congestion and the continuous advancement of intelligent transportation solution technology. Promote the development of my country's intelligent transportation.

在本发明的一种优选实施例中，管理端电脑内具有缩微车远程监控管理系统，该缩微车远程监控管理系统包括智能车终端管理模块和远程监控功能模块，其中，智能车终端管理模块包括系统基础属性配置模块、终端管理模块、状态信息基础属性配置模块、状态信息定制管理模块、控制指令发布模块和采集图片管理模块，缩微车通过服务器分别与系统基础属性配置模块、终端管理模块、状态信息基础属性配置模块、状态信息定制管理模块、控制指令发布模块和采集图片管理模块相连，系统基础属性配置模块用于设置缩微车系统的基础参数，具体包括但不限于缩微车上摄像头位置坐标标定、每秒读取摄像头图片帧数、通信接口、缩微车网络编号等信息。状态信息基础属性配置模块与缩微车的主板相连，用于管理缩微车行驶过程中的动态信息，包括行驶速度、加减速度、上下坡角度识别、行驶位置坐标等信息。状态信息定制管理模块与缩微车主板相连，用于管理需要和远程管理端的消息定制，用于封装并实时发送给缩微车的远程管理参数。控制指令发布模块用于配置所有接收和发送指令的格式及具体参数。采集图片管理模块用于管理视觉传感器在行驶过程是实时采集的道路信息，对图片进行相关分析处理，并将结果反馈给状态信息基础属性配置模块。远程监控功能模块包括用户连接状态监控模块和自动返回状态或更新信息模块，其中，用户连接状态监控模块与缩微车主板相连，用于实现每台缩微车控制系统与远程管理端电脑的连接监控，显示每台缩微车与管理端电脑的连接情况，显示每台缩微车当前的速度及运行状态等参数，自动返回状态或更新信息模块与缩微车主板相连，用于实时获取缩微车运行状态等参数，首先发送读取指令，缩微车接收到读取指令后将相关信息发送给远程控制端，远程控制端接收到最新的运动状态参数后对数据进行更新，再通过用户连接状态监控模块进行显示。In a preferred embodiment of the present invention, the management terminal computer has a remote monitoring and management system for miniature cars, and the remote monitoring and management system for miniature cars includes a smart car terminal management module and a remote monitoring function module, wherein the smart car terminal management module includes System basic attribute configuration module, terminal management module, status information basic attribute configuration module, status information customization management module, control instruction release module and image collection management module, the miniature car communicates with the system basic attribute configuration module, terminal management module, status through the server respectively The basic information attribute configuration module, the status information customization management module, the control instruction release module and the image collection management module are connected. The system basic attribute configuration module is used to set the basic parameters of the miniature car system, including but not limited to the camera position coordinate calibration on the miniature car , Read the camera picture frame number, communication interface, microcar network number and other information per second. The state information basic attribute configuration module is connected to the main board of the miniature car, and is used to manage the dynamic information during the driving process of the miniature car, including driving speed, acceleration and deceleration, uphill and downhill angle recognition, driving position coordinates and other information. The status information customization management module is connected with the miniature car main board, and is used for management needs and message customization of the remote management terminal, and is used for encapsulating and sending the remote management parameters to the miniature car in real time. The control instruction issuing module is used to configure the format and specific parameters of all receiving and sending instructions. The image collection management module is used to manage the road information collected by the visual sensor in real time during the driving process, analyze and process the images, and feed back the results to the state information basic attribute configuration module. The remote monitoring function module includes a user connection status monitoring module and an automatic return status or update information module. Among them, the user connection status monitoring module is connected with the microcar main board to realize the connection monitoring between each microcar control system and the remote management terminal computer. Display the connection between each miniature car and the management computer, display the current speed and running status of each miniature car, and automatically return to the status or update the information module. , first send a read command, and after receiving the read command, the microcar sends relevant information to the remote control terminal, and the remote control terminal updates the data after receiving the latest motion state parameters, and then displays it through the user connection status monitoring module.

本发明通过缩微车远程监控管理系统实时监控缩微车的运行状态，缩微车终端与服务器保持连接，实现在信息网络平台上根据不同的功能需求对缩微车属性信息和静、动态信息进行提取和有效利用，并根据不同的功能需求对缩微车的运行状态进行有效的监管和提供综合服务。The present invention monitors the running state of the miniature car in real time through the remote monitoring and management system of the miniature car, and the terminal of the miniature car is kept connected with the server, so that the attribute information and static and dynamic information of the miniature car can be extracted and effectively processed according to different functional requirements on the information network platform. According to different functional requirements, it can effectively supervise the running status of the miniature car and provide comprehensive services.

本发明的巡线方法包括如下步骤：Line inspection method of the present invention comprises the steps:

S1：图像获取模块获取路面的RGB彩色图像，所述RGB彩色图像中含有车道线信息；S1: the image acquisition module acquires an RGB color image of the road surface, and the RGB color image contains lane line information;

S2：图像处理模块将所述RGB彩色图像转化为灰度图像；S2: the image processing module converts the RGB color image into a grayscale image;

S3：二值化模块获取所述灰度图像中每帧图像的最佳动态阈值并进行图像分割，得到二值图像，将车道线分离出来；S3: The binarization module obtains the best dynamic threshold value of each frame image in the grayscale image and performs image segmentation to obtain a binary image and separate the lane lines;

S4：边缘检测模块对所述二值化图像进行边缘检测，得到含有车道线的内、外边缘的边缘图像；S4: The edge detection module performs edge detection on the binarized image to obtain an edge image containing the inner and outer edges of the lane line;

S5：处理器利用霍夫变换在所述边缘图像中检测车道线并获取车道线参数，建立车道线模型；S5: The processor uses Hough transform to detect lane lines in the edge image and obtain lane line parameters to establish a lane line model;

S6：处理器利用获取的所述车道线参数，通过逆透视变换获取车辆在世界坐标系中所处的车道位置数据，根据所述路面图像信息，判别车辆的行驶模式；S6: The processor uses the obtained lane line parameters to obtain the lane position data of the vehicle in the world coordinate system through inverse perspective transformation, and judge the driving mode of the vehicle according to the road surface image information;

S7：处理器利用步骤S6获取的车辆转角以及距离参数，根据车辆的行驶模式，采用分段式自适应控制策略，向控制器发送控制命令；S7: the processor uses the vehicle corner and distance parameters obtained in step S6, and according to the driving mode of the vehicle, adopts a segmented adaptive control strategy to send a control command to the controller;

S8：所述控制器接收到处理器的控制命令，通过对舵机、电机进行参数调整，实时控制车辆的行车方向以及行车速度。S8: The controller receives a control command from the processor, and controls the driving direction and driving speed of the vehicle in real time by adjusting the parameters of the steering gear and the motor.

在本实施方式中，所述步骤S5中建立的车道线模型，在车道为直道的情况下，车道线为直线模型；在车道为弯道的情况下，车道线为弯道的切线模型。In this embodiment, the lane line model established in step S5 is a straight line model when the lane is a straight road; and a tangent model of the curve when the lane is a curve.

在本实施方式中，所述步骤S6中的行驶模式包括直道模式，弯道模式，上坡模式，下坡模式，十字路口模式，丁字路口模式，跟随模式，换道模式，停车模式九种。In this embodiment, the driving modes in step S6 include straight road mode, curve mode, uphill mode, downhill mode, intersection mode, T-junction mode, following mode, lane changing mode, and parking mode.

在本发明的一种优选实施方式中，该缩微车控制方法具体包括如下步骤：In a preferred embodiment of the present invention, the microcar control method specifically includes the following steps:

第一步：搭建本发明的基于机器视觉的缩微车控制系统。该缩微车具有自动车道线跟踪、道路位置检测、自动换道、多车互动和分析车辆目标方向的能力。该缩微车控制系统基于局域无线网络远程控制，具有远程车辆跟踪、车间通信和远程行驶状态分析的能力。在本实施方式中，按照缩微智能车与真车1:10的缩微比，根据车身的高度，设置缩微智能车上摄像头的位置，使其具有最佳盲区（车头与获取图像下边缘的距离），在本实施方式中，取为28cm。The first step: build the microcar control system based on machine vision of the present invention. The miniature car has the ability of automatic lane line tracking, road position detection, automatic lane changing, multi-vehicle interaction and analysis of vehicle target direction. The miniature car control system is based on local area wireless network remote control, and has the capabilities of remote vehicle tracking, inter-vehicle communication and remote driving status analysis. In this embodiment, according to the miniature ratio of 1:10 between the miniature smart car and the real car, according to the height of the body, the position of the camera on the miniature smart car is set so that it has the best blind spot (the distance between the front of the car and the lower edge of the captured image) , in this embodiment, it is taken as 28cm.

本发明综合运用了机器视觉、人工智能、模式识别、无线传感网和仪表可靠性等多学科交叉的先进技术，针对城市交通特点，利用仿真技术，在三维立体交通沙盘仿真测试平台上复现现实交通运行状况，或虚拟出未来交通运行的状况，使得能够低成本、低危险地显现已发生或未发生的交通事件，对其特征和规律进行研究，将本发明的技术方案移植到真车上，可以帮助企业开发具有完全自主知识产权的智能汽车，为司机提供了控制车辆和预防危险情况的驾驶辅助手段，提升驾驶人员的车辆控制能力，预防交通事故和保护行人安全。The present invention comprehensively uses advanced interdisciplinary technologies such as machine vision, artificial intelligence, pattern recognition, wireless sensor network and instrument reliability, etc., aiming at the characteristics of urban traffic, using simulation technology to reproduce on the three-dimensional traffic sand table simulation test platform Actual traffic operation conditions, or virtual future traffic operation conditions, make it possible to display traffic events that have occurred or have not occurred at low cost and low risk, study their characteristics and laws, and transplant the technical solution of the present invention to real vehicles In terms of technology, it can help companies develop smart cars with completely independent intellectual property rights, provide drivers with driving assistance means to control vehicles and prevent dangerous situations, improve drivers' vehicle control capabilities, prevent traffic accidents and protect pedestrian safety.

本发明智能车辆驾驶主要研究整体自动或者作为辅助驾驶系统完成车辆驾驶任务。这些任务包括跟踪道路，保持车辆行驶在正确的道路上，维持车辆之间的一个安全距离，根据当前的交通状况和道路特征调节车辆的速度，横跨车道以达到超车和避障的目的以及找到达目的地的最短路径和在市区内方便的行驶和停靠。基于机器视觉的缩微智能车在实现对障碍物、交红绿灯、交通标志等的识别中从而达到无人驾驶的目的，都要通过机器视觉来进行实现。机器视觉就是用机器代替人眼来做测量和判断。The intelligent vehicle driving of the present invention mainly studies the overall automation or completes the vehicle driving task as an auxiliary driving system. These tasks include following the road, keeping the vehicle on the correct road, maintaining a safe distance between vehicles, adjusting the speed of the vehicle according to the current traffic conditions and road characteristics, crossing the lane to achieve the purpose of overtaking and avoiding obstacles, and finding The shortest route to the destination and convenient driving and stopping in the urban area. Miniature smart cars based on machine vision can achieve the purpose of unmanned driving in the recognition of obstacles, traffic lights, traffic signs, etc., all through machine vision. Machine vision is the use of machines instead of human eyes for measurement and judgment.

在本实施方式中，视觉传感器将被摄取目标转换成图像信号，传送给主板，该主板还包括模数转换装置，模数转换装置根据图像信号的像素分布和亮度、颜色等信息，将图像信号转变成数字化信号；主板的寻线模块，障碍物检测模块，红绿灯检测模块，交通标志检测模块，地面标志检测模块对这些数字信号进行运算来抽取各自的目标特征并根据判别的结果来控制现场的设备动作。在本实施方式中，优选采用OpenCV图像处理软件内的函数对图像进行处理。在本实施方式中，采用的函数包括但不限于图片格式转换函数cvCvtColor()，设置图像感兴趣区域函数cvSetImageROI()，二值化处理函数cvThreshold()，寻找轮廓函数cvFindContours()，轮廓边界框返回函数cvBoundingRect()等。In this embodiment, the visual sensor converts the captured object into an image signal and transmits it to the main board. The main board also includes an analog-to-digital conversion device. The analog-to-digital conversion device converts the image signal to Transformed into digital signals; the main board's line-hunting module, obstacle detection module, traffic light detection module, traffic sign detection module, and ground sign detection module perform operations on these digital signals to extract their respective target features and control the scene based on the results of the discrimination. Device action. In this embodiment, it is preferable to use the functions in the OpenCV image processing software to process the image. In this embodiment, the functions used include but are not limited to the image format conversion function cvCvtColor(), the image region of interest function cvSetImageROI(), the binarization processing function cvThreshold(), the contour finding function cvFindContours(), and the contour bounding box Return function cvBoundingRect() etc.

第二步：主板的初始化模块对视觉传感器、马达和舵机进行初始化。Step 2: The initialization module of the main board initializes the vision sensor, motor and steering gear.

初始化该模块可实现的功能是：对所有模块中参数的初始化，包括但不限于对视觉传感器属性，投影矩阵，马达速度，舵机的方向进行初始化，在本发明的一个优选实施方式中，初始化的取值可以根据实验或者本领域常用数据进行设定。在具体算法实现上，主要调用opencv的函数进行处理。例如设置视觉传感器属性参数调用的函数包括：The functions that can be realized by initializing this module are: initialization of parameters in all modules, including but not limited to initialization of visual sensor properties, projection matrix, motor speed, direction of steering gear, in a preferred embodiment of the present invention, initialization The value of can be set according to experiments or commonly used data in the field. In terms of specific algorithm implementation, the function of opencv is mainly called for processing. For example, the functions called by setting the visual sensor property parameters include:

cvCreateCameraCapture();该函数为摄像头获取函数,可由该函数获得缩微智能车上下两个摄像头的属性值。cvCreateCameraCapture(); This function is a camera acquisition function, which can be used to obtain the property values of the upper and lower cameras of the miniature smart car.

cvSetCaptureProperty(pCapture,CV_CAP_PROP_FRAME_WIDTH,320);cvSetCaptureProperty(pCapture,CV_CAP_PROP_FRAME_WIDTH,320);

cvSetCaptureProperty(pCapture,CV_CAP_PROP_FRAME_HEIGHT,240);cvSetCaptureProperty(pCapture,CV_CAP_PROP_FRAME_HEIGHT,240);

这两个函数是对摄像头获取的图片的尺寸进行设定，函数中设定的图片尺寸为320像素*240像素的大小。These two functions are to set the size of the picture captured by the camera, and the size of the picture set in the function is 320 pixels*240 pixels.

第三步：视觉传感器获取路面图片并将路面图片传输给主板。Step 3: The vision sensor acquires the road surface picture and transmits the road surface picture to the main board.

第四步：主板根据路面图片进行寻线、寻线方法包括如下步骤：Step 4: The main board searches for the line according to the road surface picture, and the line-hunting method includes the following steps:

S1：图像获取模块获取路面的RGB彩色图像，所述RGB彩色图像中含有车道线信息，通常道路上车道线的颜色一般为白色，或间断，或连续；道路路面为黑灰色，路宽约为35cm，本实施方式中暂不考虑下雨天雨水反光等因素的影响；S1: The image acquisition module acquires the RGB color image of the road surface, which contains lane line information. Usually, the color of the lane line on the road is white, or intermittent, or continuous; the road surface is black and gray, and the road width is about 35cm, the impact of factors such as rain reflection in rainy days is not considered in this embodiment;

S2：图像处理模块将所述RGB彩色图像转化为灰度图像；S2: the image processing module converts the RGB color image into a grayscale image;

S3：二值化模块运用大津法获取所述灰度图像中每帧图像的最佳动态阈值并进行图像分割，得到二值图像，将车道线分离出来；S3: The binarization module uses the Otsu method to obtain the best dynamic threshold of each frame image in the grayscale image and performs image segmentation to obtain a binary image and separate the lane lines;

S4：边缘检测模块运用canny算子对所述二值化图像进行边缘检测，得到含有车道线的内、外边缘的边缘图像；S4: The edge detection module uses the canny operator to perform edge detection on the binarized image to obtain an edge image containing the inner and outer edges of the lane line;

S5：处理器利用霍夫变换在所述边缘图像中检测车道线并获取车道线参数，建立车道线模型，在车道为直道的情况下，车道线为直线模型；在车道为弯道的情况下，车道线为弯道的切线模型；S5: The processor uses the Hough transform to detect the lane line in the edge image and obtain the lane line parameters to establish a lane line model. When the lane is a straight road, the lane line is a straight line model; when the lane is a curved road , the lane line is the tangent model of the curve;

S6：处理器利用获取的所述车道线参数，通过逆透视变换获取车辆在世界坐标系中所处的车道位置数据，并辅以图像中其他特征的提取，判别车辆当前时刻的状态模式，如所处车道为直道或弯道，是否进入十字路口或丁字路口等，以便后续控制车辆实时巡线；S6: The processor uses the obtained lane line parameters to obtain the lane position data of the vehicle in the world coordinate system through inverse perspective transformation, supplemented by the extraction of other features in the image, and judge the state mode of the vehicle at the current moment, such as The lane you are in is a straight road or a curve, whether you are entering a crossroad or a T-junction, etc., so that you can follow up and control the vehicle to patrol the line in real time;

S7：处理器利用步骤S6获取的车辆转角以及距离参数，根据车辆的行驶模式，采用分段式自适应控制策略，向控制器发送控制命令；S7: the processor uses the vehicle corner and distance parameters obtained in step S6, and according to the driving mode of the vehicle, adopts a segmented adaptive control strategy to send a control command to the controller;

S8：所述控制器接收到处理器的控制命令，通过对舵机、电机进行参数调整，实时控制车辆的行车方向以及行车速度。S8: The controller receives a control command from the processor, and controls the driving direction and driving speed of the vehicle in real time by adjusting the parameters of the steering gear and the motor.

本发明的巡线方法稳定、易控制，巡线时间可以在10毫秒之内完成，缩微智能车的行驶速度可达1-2米/秒，折算到真车约为40-60km/h。The line inspection method of the present invention is stable and easy to control, and the line inspection time can be completed within 10 milliseconds, and the travel speed of the miniature smart car can reach 1-2 m/s, which is about 40-60 km/h when converted to a real car.

在本实施方式中，处理器也可以利用图像变换函数hLines2()进行找线，图像变换函数cvHoughLines2()找到了图像中的许多线，有些是想要的，有些是不想要的，为了得到车道线，就必须要进行条件筛选，具体条件可以包括但不限于对车道线的距离、斜率阈值，车道线的距离和斜率阈值可以根据具体现实中实际道路上的车道线的距离和车道线的斜率进行选取，也可以按比例进行减小或者放大选取。In this embodiment, the processor can also use the image transformation function hLines2() to find lines. The image transformation function cvHoughLines2() has found many lines in the image, some of which are desired and some are not. In order to obtain the lane line, it is necessary to filter the conditions, the specific conditions can include but not limited to the distance of the lane line, the slope threshold, the distance of the lane line and the slope threshold can be based on the distance of the lane line and the slope of the lane line on the actual road in the specific reality Selection can also be made proportionally to reduce or enlarge the selection.

找到车道线以后，为了方便缩微车换道，可以根据车道线左右是否存在绿地的条件来确定所处的具体是哪个车道，具体的换道条件为，如果车道右侧有绿地，则说明车道是最右面的车道，只能向左换车道；如果车道左侧有绿地，则说明车道是做左侧的绿地，只能向右换车道，如果左右两侧都没有绿地，则说明是中间的车道，则向左侧或者右侧换车道都可以。本发明的寻线方法能够快速准确地找到车道线，提高了行车的安全性。After finding the lane line, in order to facilitate the miniature car to change lanes, you can determine which lane you are in according to whether there is a green space around the lane line. The specific lane change conditions are, if there is a green space on the right side of the lane, it means that the lane is The rightmost lane can only change lanes to the left; if there is a green space on the left side of the lane, it means that the lane is a green space on the left and can only change lanes to the right; if there is no green space on the left and right sides, it means that it is the middle lane , you can change lanes to the left or right. The line-finding method of the invention can quickly and accurately find lane lines, thereby improving driving safety.

在本发明的一种优选实施方式中，障碍物检测模块对前方出现的障碍物进行识别以及作出相应的处理，障碍物检测方法包括如下步骤：In a preferred embodiment of the present invention, the obstacle detection module identifies the obstacles appearing ahead and performs corresponding processing, and the obstacle detection method includes the following steps:

S21:障碍物检测模块接收视觉传感器获取的路面图片后对路面图片进行格式转换。障碍物检测模块从视觉传感器获取图片后，为了能避免光线影响，在图片的处理过程中需要选择合适的颜色空间，在本实施方式中，将RGB格式转换为HSV格式来进行处理，采用的格式转换函数为：cvCvtColor(image,imgHSV,CV_RGB2HSV)。S21: The obstacle detection module converts the format of the road surface image after receiving the road surface image acquired by the visual sensor. After the obstacle detection module acquires the picture from the visual sensor, in order to avoid the influence of light, it needs to select an appropriate color space during the process of picture processing. In this embodiment, the RGB format is converted to HSV format for processing. The format used The conversion function is: cvCvtColor(image,imgHSV,CV_RGB2HSV).

S22：障碍物检测模块进行灰度转换、二值化处理、寻找轮廓和条件筛选。在本实施方式中，调用opencv中库函数对路面图片进行灰度转换、二值化处理、寻找轮廓、条件筛选。具体主要调用的opencv函数为：S22: The obstacle detection module performs gray scale conversion, binarization processing, contour finding and condition screening. In this embodiment, the library function in opencv is called to perform grayscale conversion, binarization processing, contour finding, and conditional screening on the road surface image. The specific opencv function that is mainly called is:

二值化处理函数cvThreshold()；Binarization processing function cvThreshold();

寻找轮廓函数cvFindContours()。Find the contour function cvFindContours().

在寻找到轮廓后，对障碍物进行条件筛选，具体根据障碍物距缩微车的距离、障碍物的颜色、尺寸、面积等特征进行筛选。便能实现对障碍物的识别，从而指挥缩微车的动作，包括停止或换道。在本实施方式中，筛选的条件可以根据实际试验进行设定。After the outline is found, the obstacles are screened according to the distance from the obstacle to the miniature car, the color, size, area and other characteristics of the obstacle. The identification of obstacles can be realized, so as to direct the actions of the miniature car, including stopping or changing lanes. In this embodiment, the screening conditions can be set according to actual experiments.

在本发明的另一种优选实施方式中，红绿灯检测方法包括如下步骤：In another preferred embodiment of the present invention, traffic light detection method comprises the following steps:

S31：当缩微车行驶至十字路口模式时，红绿灯检测模块首先判断道路图片是否存在停止线，若存在，则执行步骤S32。S31: When the miniature car travels to the intersection mode, the traffic light detection module first judges whether there is a stop line in the road image, and if so, executes step S32.

S32：根据视觉传感器与红绿灯的高度阈值，获取感兴趣区域。在红绿灯的检测过程中，由于摄像头与红绿灯的高度均是固定的，为了提高处理速度，减少环境干扰等因素，因此采用设置感兴趣区域。在本实施方式中，视觉传感器与红绿灯的高度阈值可以根据具体试验或者实际道路中的视觉传感器与红绿灯的高度进行设置，具体设置的高度阈值为红绿灯的高度加减一定的度所形成的范围。S32: Obtain the region of interest according to the height threshold of the visual sensor and the traffic light. In the detection process of traffic lights, since the heights of the camera and the traffic lights are fixed, in order to improve the processing speed and reduce environmental interference and other factors, the area of interest is set. In this embodiment, the height threshold of the visual sensor and the traffic light can be set according to the height of the visual sensor and the traffic light in a specific test or an actual road, and the specifically set height threshold is the range formed by adding or subtracting a certain degree to the height of the traffic light.

S33：读取感兴趣区域内的像素点的R、G、B三刺激值并与设定的红绿灯的R、G、B三刺激值进行比较，当满足误差要求时，感兴趣区域为目标区域。具体的误差范围可以根据具体实验具体设定。S33: Read the R, G, and B tri-stimulus values of the pixels in the region of interest and compare them with the set R, G, and B tri-stimulus values of the traffic lights. When the error requirements are met, the region of interest is the target region . The specific error range can be specifically set according to specific experiments.

S34：对目标区域进行条件筛选，所述条件筛选的项目包括筛选像素点的个数，R、G、B三刺激值分别所占的比例，当所有条件均满足，则判断出是红灯还是绿灯。S34: Carry out conditional screening on the target area, the items of the conditional screening include the number of screened pixels, the proportions of the R, G, and B tri-stimulus values, and when all the conditions are met, it is judged whether it is a red light or a red light. green light.

本发明的红绿灯检测方法通过选定感兴趣区域，并在感兴趣区域中选择目标区域，提高了红绿灯检测的快速性，对目标区域进行条件筛选，提高了红绿灯检测的准确性。The traffic light detection method of the present invention improves the rapidity of traffic light detection by selecting an area of interest and selects a target area in the interest area, and performs conditional screening on the target area to improve the accuracy of traffic light detection.

在本发明的一种优选实施方式中，交通标志类型包括直行，禁止直行，右转，禁止右转，左转，禁止左转六种类型，缩微智能车通过该模块实现对以上交通标志的识别与判断。交通标志检测方法包括如下步骤：In a preferred embodiment of the present invention, the types of traffic signs include six types: go straight, no go straight, turn right, no right turn, left turn, no left turn, and the miniature smart car realizes the identification of the above traffic signs through this module with judgment. The traffic sign detection method includes the following steps:

S41：所述交通标志检测模块检测所获取的图片中某个像素点及其周围连通域是否存在红色像素点，若存在，则设置为目标区域并进入步骤S42，具体的联通域的大小可以根据试验中交通标志的大小设定，具体可以但不限于小于、大于或等于交通标志的大小。S41: The traffic sign detection module detects whether there are red pixels in a certain pixel point in the acquired picture and its surrounding connected domains. If there are red pixels, set it as the target area and enter step S42. The size of the specific connected domain can be determined according to The size setting of the traffic sign in the experiment can be, but not limited to, smaller than, larger than or equal to the size of the traffic sign.

S42：在目标区域内搜索黑色像素点，并获取包含黑色像素的连通区域的矩形，通过尺寸调整函数调整所述矩形的大小，在本实施方式中，通过尺寸调整函数cvResize()将矩形大小调整为7个像素*5个像素大小的矩形。S42: Search for black pixels in the target area, and obtain a rectangle of a connected area containing black pixels, and adjust the size of the rectangle through a size adjustment function. In this embodiment, adjust the size of the rectangle through the size adjustment function cvResize() It is a rectangle with a size of 7 pixels*5 pixels.

S43：提取所述矩形内的特征信息，与模板进行匹配，如果匹配成功，则得出交通标志的类型，使缩微车将执行相应的操作。S43: Extract the feature information in the rectangle, and match it with the template. If the matching is successful, obtain the type of traffic sign, and make the miniature car perform a corresponding operation.

本发明的交通标志检测方法通过选定目标区域并调整目标区域内矩形的大小，提高了交通标志检测的快速性，将特征信息与模板进行匹配，提高了交通标志检测的准确性。The traffic sign detection method of the present invention improves the speed of traffic sign detection by selecting the target area and adjusting the size of the rectangle in the target area, and matches the characteristic information with the template to improve the accuracy of traffic sign detection.

在本发明的一种优选实施方式中，实现缩微车对所处车道地面标志的识别及反应行为，如直走右转标志，地面标志检测方法包括如下步骤：In a preferred embodiment of the present invention, the recognition and reaction behavior of the miniature car to the ground signs of the lane, such as going straight and turning right, the ground sign detection method includes the following steps:

S51：地面标志检测模块接收视觉传感器获取的路面图片后对路面图片进行灰度转换以及二值化处理。S51: The ground mark detection module performs grayscale conversion and binarization processing on the road surface image after receiving the road surface image acquired by the visual sensor.

S52：地面标志检测模块对二值化图像进行边缘检测，具体可以但不限于采用Canny函数二值化图像进行边缘检测。S52: The ground mark detection module performs edge detection on the binarized image, specifically, but not limited to, using the Canny function binarized image to perform edge detection.

S53：地面标志检测模块进行找线处理，并根据线条的大小进行筛选。S53: The ground mark detection module performs line finding processing, and performs screening according to the size of the lines.

S54：提取所述线条的特征信息，与模板进行匹配，如果匹配成功，则得出地面标志的类型，使缩微车将执行相应的操作。S54: Extract the feature information of the line, and match it with the template. If the match is successful, obtain the type of the ground mark, so that the miniature vehicle will perform a corresponding operation.

在本发明的一个优选实施方式中，主板的控制程序的运行过程为：首先，主板的初始化模块对视觉传感器、马达和舵机进行初始化。初始化后，视觉传感器检测路面图片并将路面图片传输给寻线模块、障碍物检测模块、红绿灯检测模块、交通标志检测模块以及地面标志检测模块。寻线模块寻找车道线，缩微车正常行进。在行进的过程中，障碍物检测模块判断车道是否干净，红绿灯检测模块检测是否用停止线。障碍物检测模块判断车道干净，则缩微车继续寻线行进，如果障碍物检测模块判断车道不干净，障碍物检测模块检测并经过条件筛选判断是否有障碍物，如果有障碍物则停车，停车后重新检测是否有障碍物，如果仍有障碍物则继续停车，如果没有障碍物则继续寻线行进，避免了错误检测障碍物导致停车的情况发生，提高了准确性；如果障碍物检测模块判断没有障碍物或者经过条件筛选后不需要停车，则判断换道条件是否符合，如果换道条件符合，则进行换道，在换道的过程不断检测换道是否结束，如果结束，则继续寻线行进，如果没结束，继续换道，如果换道条件不满足，则在原车道继续行驶。In a preferred embodiment of the present invention, the running process of the control program of the main board is as follows: firstly, the initialization module of the main board initializes the visual sensor, the motor and the steering gear. After initialization, the visual sensor detects the road surface picture and transmits the road surface picture to the line hunting module, obstacle detection module, traffic light detection module, traffic sign detection module and ground sign detection module. The line-finding module looks for the lane line, and the miniature car moves normally. During the driving process, the obstacle detection module judges whether the lane is clean, and the traffic light detection module detects whether the stop line is used. The obstacle detection module judges that the lane is clean, and the miniature car continues to search for the line. If the obstacle detection module judges that the lane is not clean, the obstacle detection module detects and judges whether there is an obstacle after conditional screening. If there is an obstacle, it stops. Re-detect whether there is an obstacle, if there is still an obstacle, continue to stop, if there is no obstacle, continue to search for the line, avoiding the situation that the wrong detection of obstacles leads to parking, and improve the accuracy; if the obstacle detection module judges that there is no Obstacles or conditions do not need to stop, then judge whether the lane change conditions are met, if the lane change conditions are met, then change lanes, and constantly check whether the lane change is over during the lane change process, if it is over, continue to search for lines , if it is not over, continue to change lanes, if the lane change conditions are not met, continue driving in the original lane.

红绿灯检测模块检测是否用停止线，如果没有停止线，则缩微车继续寻线行进，如果有停止线，则判断是否有红灯；如果有红灯，则停车，红灯灭后，进入十字路口行进模式，寻线模块寻找车道线，寻找到车道线后，缩微车正常行进，如果没有寻找到车道线，寻线模块继续寻线；如果没有红灯，交通标志检测模块检测是否有交通标志，如果有，则按照交通标志行走，寻线模块寻找车道线，寻找到车道线后，缩微车正常行进，如果没有寻找到车道线，寻线模块继续寻线。如果没有交通标志，则进入十字路口行进模式，寻线模块寻找车道线，寻找到车道线后，缩微车正常行进，如果没有寻找到车道线，寻线模块继续寻线。The traffic light detection module detects whether the stop line is used. If there is no stop line, the miniature car will continue to search for the line. If there is a stop line, it will judge whether there is a red light; if there is a red light, stop. After the red light goes out, enter the intersection In the driving mode, the line hunting module searches for the lane line. After finding the lane line, the miniature car moves normally. If no lane line is found, the line hunting module continues to find the line; if there is no red light, the traffic sign detection module detects whether there is a traffic sign. If there is, then walk according to the traffic sign, and the line-hunting module searches for the lane line. After finding the lane line, the miniature car proceeds normally. If no lane line is found, the line-finding module continues to find the line. If there is no traffic sign, then enter the intersection traveling mode, and the line-finding module searches for the lane line. After finding the lane line, the miniature car moves normally. If no lane line is found, the line-finding module continues to find the line.

第五步：主板向控制板下达控制命令，控制马达和舵机的运行。具体是主板的速度控制模块接收初始化模块、寻线模块、障碍物检测模块、红绿灯检测模块、交通标志检测模块、地面标志检测模块检测的信息并产生速度控制信息，所述速度控制模块将速度控制信息传输给所述控制板，所述控制板控制所述马达的运行速度。所述方向控制模块接收所述初始化模块、寻线模块、障碍物检测模块、红绿灯检测模块、交通标志检测模块、地面标志检测模块检测的信息，并产生方向控制信息，所述方向控制模块将方向控制信息传输给所述控制板，所述控制板控制所述舵机的方向。Step 5: The main board issues control commands to the control board to control the operation of the motor and steering gear. Specifically, the speed control module of the main board receives the information detected by the initialization module, the line hunting module, the obstacle detection module, the traffic light detection module, the traffic sign detection module, and the ground sign detection module and generates speed control information, and the speed control module controls the speed. Information is transmitted to the control board, which controls the operating speed of the motor. The direction control module receives the information detected by the initialization module, the line hunting module, the obstacle detection module, the traffic light detection module, the traffic sign detection module, and the ground sign detection module, and generates direction control information, and the direction control module controls the direction The control information is transmitted to the control board, and the control board controls the direction of the steering gear.

第六步：主板将缩微车的运行信息传输给服务器并通过服务器传输给管理端电脑，管理端电脑通过缩微车远程监控管理系统实现对缩微车的运行进行监控管理。Step 6: The main board transmits the running information of the miniature car to the server and transmits it to the management computer through the server. The management computer realizes the monitoring and management of the operation of the miniature car through the remote monitoring and management system of the miniature car.

在本说明书的描述中，参考术语“一个实施例”、“一些实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本发明的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不一定指的是相同的实施例或示例。而且，描述的具体特征、结构、材料或者特点可以在任何的一个或多个实施例或示例中以合适的方式结合。In the description of this specification, descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific examples", or "some examples" mean that specific features described in connection with the embodiment or example , structure, material or characteristic is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

尽管已经示出和描述了本发明的实施例，本领域的普通技术人员可以理解：在不脱离本发明的原理和宗旨的情况下可以对这些实施例进行多种变化、修改、替换和变型，本发明的范围由权利要求及其等同物限定。Although the embodiments of the present invention have been shown and described, those skilled in the art can understand that various changes, modifications, substitutions and modifications can be made to these embodiments without departing from the principle and spirit of the present invention. The scope of the invention is defined by the claims and their equivalents.

Claims (5)

Translated fromChinese

1.一种基于机器视觉的车辆避障方法，其特征在于，包括如下步骤：1. a vehicle obstacle avoidance method based on machine vision, is characterized in that, comprises the steps:S1：处理器采用基于单幅图像的障碍物检测算法判定缩微车前方是否存在障碍物，如果有障碍物，则将障碍物距离和障碍物所处车道的检测信息传输给控制器，并执行步骤S2，如果没有障碍物，则继续执行步骤S1；S1: The processor uses an obstacle detection algorithm based on a single image to determine whether there is an obstacle in front of the miniature car. If there is an obstacle, it transmits the detection information of the obstacle distance and the lane where the obstacle is located to the controller, and executes the steps S2, if there is no obstacle, proceed to step S1;S2：处理器对两个摄像头进行标定，采用立体视觉的方法判定障碍物中心位置、上边界中心位置的三维坐标，从而确定障碍物的高度并将所述障碍物高度信息传输给控制器；S2: The processor calibrates the two cameras, and uses a stereo vision method to determine the three-dimensional coordinates of the center position of the obstacle and the center position of the upper boundary, thereby determining the height of the obstacle and transmitting the height information of the obstacle to the controller;S3：车身两侧的测距装置检测相邻两车道的车辆状况，为避障换道提供可行驶的区域并将所述可行使区域的信息传输给所述控制器；S3: The distance measuring devices on both sides of the vehicle body detect the vehicle conditions of the adjacent two lanes, provide a drivable area for obstacle avoidance and lane change, and transmit the information of the drivable area to the controller;S4：控制器根据获得的障碍物距离、障碍物所处车道、障碍物高度以及相邻两车道的可行使区域信息，采用自适应换道策略，向运行控制模块发送命令，控制车轮的转向和速度，完成车辆的自主换道。S4: The controller adopts an adaptive lane-changing strategy based on the obtained obstacle distance, the lane where the obstacle is located, the height of the obstacle, and the maneuverable area information of the two adjacent lanes, and sends commands to the operation control module to control the steering and steering of the wheels. speed to complete the autonomous lane change of the vehicle.2.如权利要求1所述的基于机器视觉的车辆避障方法，其特征在于：所述单幅图像的障碍物检测算法的步骤为：2. the vehicle obstacle avoidance method based on machine vision as claimed in claim 1, is characterized in that: the step of the obstacle detection algorithm of described single image is:S21：对图像进行色彩空间变换RGB转HSV，并单独抽取S通道图像；S21: Perform color space transformation on the image from RGB to HSV, and separately extract the S channel image;S22：对S图像利用大津法进行动态阈值二值化得到二值图像Bin，对Bin图像去噪，消除干扰；S22: Perform dynamic threshold binarization on the S image using the Otsu method to obtain a binary image Bin, denoise the Bin image, and eliminate interference;S23：在Bin图像上搜索所有可能存在的障碍物轮廓，根据设置的筛选条件将误判的障碍物剔除；S23: Search for all possible obstacle contours on the Bin image, and remove misjudged obstacles according to the set filter conditions;S24：并根据车道线的方位判定障碍物所处车道。S24: and determine the lane where the obstacle is located according to the orientation of the lane line.3.如权利要求1所述的基于机器视觉的车辆避障方法，其特征在于：所述立体视觉的方法为：3. the vehicle obstacle avoidance method based on machine vision as claimed in claim 1, is characterized in that: the method of described stereo vision is:S31：根据主摄像头A和辅助摄像头B的安装位置，分别进行标定，得出摄像头的内参数、外参数，以及主摄像头A和辅助摄像头B之间的间距b；S31: According to the installation positions of the main camera A and the auxiliary camera B, perform calibration respectively to obtain the internal parameters and external parameters of the cameras, and the distance b between the main camera A and the auxiliary camera B;S32：对辅助摄像头B获取的图像采用基于单幅图像的障碍物检测方法，获取障碍物中心位置、上边界中心位置在图像坐标系下的二维坐标；S32: Using an obstacle detection method based on a single image for the image acquired by the auxiliary camera B, acquire the two-dimensional coordinates of the center position of the obstacle and the center position of the upper boundary in the image coordinate system;S33：利用如下公式获取障碍物在主摄像头坐标系下的三维坐标x_c,y_c,z_c，并利用摄像头的标定参数，得出障碍物在世界坐标系下的三维坐标，S33: Use the following formula to obtain the three-dimensional coordinates x_c , y_c , z_c of the obstacle in the main camera coordinate system, and use the calibration parameters of the camera to obtain the three-dimensional coordinates of the obstacle in the world coordinate system,$<span><span>(</span>(</span>{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>c</span>c</span>}}<span><span>,</span>,</span>{\mathrm{<span><span>y</span>the\; y</span>}}_{\mathrm{<span><span>c</span>c</span>}}<span><span>,</span>,</span>{\mathrm{<span><span>z</span>z</span>}}_{\mathrm{<span><span>c</span>c</span>}}<span><span>)</span>)</span><span><span>=</span>=</span><span><span>(</span>(</span>\frac{{\mathrm{<span><span>bu</span>bu</span>}}_{\mathrm{<span><span>1</span>1</span>}}}{{\mathrm{<span><span>u</span>u</span>}}_{\mathrm{<span><span>1</span>1</span>}}<span><span>-</span>-</span>{\mathrm{<span><span>u</span>u</span>}}_{\mathrm{<span><span>2</span>2</span>}}}<span><span>,</span>,</span>\frac{{\mathrm{<span><span>bv</span>bv</span>}}_{\mathrm{<span><span>1</span>1</span>}}}{{\mathrm{<span><span>u</span>u</span>}}_{\mathrm{<span><span>1</span>1</span>}}<span><span>-</span>-</span>{\mathrm{<span><span>u</span>u</span>}}_{\mathrm{<span><span>2</span>2</span>}}}<span><span>,</span>,</span>\frac{\mathrm{<span><span>bf</span>b\; f</span>}}{{\mathrm{<span><span>u</span>u</span>}}_{\mathrm{<span><span>1</span>1</span>}}<span><span>-</span>-</span>{\mathrm{<span><span>u</span>u</span>}}_{\mathrm{<span><span>2</span>2</span>}}}<span><span>)</span>)</span><span><span>,</span>,</span>$其中，b为两摄像头的间距，f为摄像头的焦距，（u₁，v₁）为标定点在A摄像头平面的投影位置，（u₂，v₂）为标定点在B摄像头平面的投影位置，并且v₁=v₂。Among them, b is the distance between the two cameras, f is the focal length of the camera, (u₁ , v₁ ) is the projection position of the calibration point on the plane of camera A, (u₂ , v₂ ) is the projection position of the calibration point on the plane of camera B , and v₁ =v₂ .4.如权利要求1所述的基于机器视觉的车辆避障方法，其特征在于：所述分段式自适应控制策略是指：根据车身距离车道中间的偏移角度和距离，以及车身此刻的行驶速度，根据参数degree_per_dist自适应适度调整下一时刻的车身转角和车速，当车辆行驶在直道的中间附近位置时，始终保持车身以全速直行的方向行驶，若偏向车道两侧的任意一侧，则进行微调。4. The vehicle obstacle avoidance method based on machine vision as claimed in claim 1, wherein: the segmented adaptive control strategy refers to: according to the offset angle and distance between the vehicle body and the middle of the lane, and the current position of the vehicle body Driving speed, according to the parameter degree_per_dist, adaptively adjust the body corner and speed at the next moment. When the vehicle is driving near the middle of the straight road, always keep the body running in the straight direction at full speed. If it is biased to either side of the lane, fine-tuning.5.一种利用权利要求1所述的基于机器视觉的车辆避障系统，其特征在于，包括主摄像头A、辅助摄像头B、测距装置、处理器和控制器，5. A vehicle obstacle avoidance system based on machine vision according to claim 1 is characterized in that, comprising a main camera A, an auxiliary camera B, a distance measuring device, a processor and a controller,所述主摄像头A和辅助摄像头B分别用于获取路面的图像并将所述图像传输给所述处理器；The main camera A and the auxiliary camera B are respectively used to acquire images of the road surface and transmit the images to the processor;所述处理器用于判定缩微车前方是否存在障碍物并将障碍物距离和障碍物所处车道的检测信息传输给控制器，同时处理器对主摄像头A和辅助摄像头B进行标定，判定障碍物中心位置、上边界中心位置的三维坐标，从而确定障碍物的高度并将所述障碍物高度信息传输给控制器；The processor is used to determine whether there is an obstacle in front of the miniature car and transmit the detection information of the obstacle distance and the lane where the obstacle is located to the controller. At the same time, the processor calibrates the main camera A and the auxiliary camera B to determine the center of the obstacle. position, the three-dimensional coordinates of the center position of the upper boundary, so as to determine the height of the obstacle and transmit the information of the height of the obstacle to the controller;所述测距装置位于车身两侧，用于检测相邻两车道的车辆状况并将可行使区域的信息传输给所述控制器；The distance measuring device is located on both sides of the vehicle body, and is used to detect the vehicle conditions in two adjacent lanes and transmit the information of the driving area to the controller;控制器与运行控制模块相连，所述运行控制模块与车轮相连，控制器根据获得的障碍物距离、障碍物所处车道、障碍物高度以及相邻两车道的可行使区域信息，向运行控制模块发送命令，所述运行控制模块控制车轮的转向和速度，完成车辆的自主换道。The controller is connected to the operation control module, and the operation control module is connected to the wheels. The controller sends information to the operation control module according to the obtained obstacle distance, the lane where the obstacle is located, the height of the obstacle, and the feasible area information of the two adjacent lanes. Sending a command, the operation control module controls the steering and speed of the wheels to complete the autonomous lane change of the vehicle.

Images