技术领域technical field
本发明涉及自动控制技术领域，特别是涉及一种基于家庭庭院的户外自动行走装置及其控制系统和方法。The invention relates to the technical field of automatic control, in particular to an outdoor automatic walking device based on a family yard and its control system and method.
背景技术Background technique
随着计算机技术和人工智能技术的不断进步，类似于智能机器人的家庭庭院户外移动机器人已经开始慢慢的走进人们的生活。目前许多科技公司均开发了全自动吸尘器并已经投入市场。这种全自动吸尘器通常体积小巧，无需人工操控，一次设置之后就无需再投入精力管理，将用户从清洁、草坪维护等枯燥且费时费力的家务工作中解放出来，因此受到极大欢迎。With the continuous advancement of computer technology and artificial intelligence technology, outdoor mobile robots in home gardens similar to intelligent robots have begun to slowly enter people's lives. At present, many technology companies have developed fully automatic vacuum cleaners and have put them into the market. This kind of automatic vacuum cleaner is usually small in size and does not require manual control. After one setting, it does not need to invest in energy management. It liberates users from boring and time-consuming housework such as cleaning and lawn maintenance, so it is very popular.
目前对家庭庭院户外移动机器人进行控制，存在以下方法：1）基于编码器的航迹推算法：方案的基本要求是车轮不打滑，并且地面平坦，这样才可以根据轮子旋转的圈数来进行推算，但对于自动割草机而言，打滑和平坦均无法充分保证，这将使得积累误差大到无法接受；2）基于多基站的绝对定位：基本方案是在区域四角设置某种发射器，如超声、红外、电磁射频等，通过类似GPS的原理进行直接定位，具有精度高等优点，对于自动割草机而言，超声信号会被植物多次反射、红外无法穿过植物且受阳光干扰、电磁成本超高；3）磁轨式定位：其需要铺设较密集的地下磁轨，对于自动割草机而言基本不适用。At present, there are the following methods to control the outdoor mobile robot in the home garden: 1) Encoder-based dead reckoning method: the basic requirement of the scheme is that the wheels do not slip and the ground is flat, so that the reckoning can be performed according to the number of turns of the wheels , but for automatic lawn mowers, slipping and flatness cannot be fully guaranteed, which will make the accumulated error unacceptably large; 2) Absolute positioning based on multiple base stations: the basic solution is to set some kind of transmitter at the four corners of the area, such as Ultrasound, infrared, electromagnetic radio frequency, etc., are directly positioned by a principle similar to GPS, which has the advantages of high precision. For automatic lawn mowers, ultrasonic signals will be reflected by plants multiple times, infrared cannot pass through plants and are interfered by sunlight, electromagnetic waves, etc. The cost is extremely high; 3) Magnetic track positioning: it needs to lay dense underground magnetic tracks, which is basically not suitable for automatic lawn mowers.
因此传统的户外移动机器人控制方法存在各种缺陷，并不能有效控制户外移动机器人的移动，需要改进。Therefore, there are various defects in the traditional outdoor mobile robot control method, and it cannot effectively control the movement of the outdoor mobile robot, which needs to be improved.
发明内容Contents of the invention
基于此，有必要针对上述问题，提供一种可有效控制的基于家庭庭院的户外自动行走装置及其控制系统和方法。Based on this, it is necessary to provide an effectively controllable outdoor self-propelled device based on a home garden and its control system and method for the above-mentioned problems.
一种基于家庭庭院的户外自动行走装置，包括：壳体和行走模块，所述行走模块安装于所述壳体上，还包括：An outdoor automatic walking device based on a family garden, comprising: a housing and a walking module, the walking module is installed on the housing, and also includes:
安装于所述壳体上的图像采集模块，用于当所述自动行走装置在工作表面上行走时，连续采集工作表面上的图像；The image acquisition module installed on the housing is used to continuously acquire images on the work surface when the automatic walking device walks on the work surface;
安装于所述壳体中的主控制模块，与所述行走模块和所述图像采集模块分别连接，用于连续执行从采集的第一图像中确定特征区域，在所述第一图像后所采集的第二图像中识别所述特征区域，根据所述特征区域的几何变形和位移计算出并记录所述自动行走装置的位置变化数据和姿态变化数据；所述主控制模块还用于根据已记录的所述位置变化数据和姿态变化数据计算得到所述自动行走装置的行走轨迹。The main control module installed in the housing is respectively connected with the walking module and the image acquisition module, and is used to continuously determine the feature area from the first image collected, and the image collected after the first image Identify the feature area in the second image of the feature area, calculate and record the position change data and attitude change data of the automatic walking device according to the geometric deformation and displacement of the feature area; the main control module is also used for recording The position change data and attitude change data are calculated to obtain the walking trajectory of the automatic walking device.
在一个实施例中，所述图像采集模块用于连续采集工作表面上的图像时，工作表面上采集区域的大小为大于或等于100平方厘米。In one embodiment, when the image acquisition module is used to continuously acquire images on the working surface, the size of the acquisition area on the working surface is greater than or equal to 100 square centimeters.
在一个实施例中，所述图像采集模块位于壳体的靠近工作表面的一侧，采集方向朝向工作表面，且所述图像采集模块与工作表面之间的距离大于或等于3厘米。In one embodiment, the image acquisition module is located on a side of the housing close to the working surface, the acquisition direction faces the working surface, and the distance between the image acquisition module and the working surface is greater than or equal to 3 cm.
在一个实施例中，所述第一图像和所述第二图像为所述图像采集模块采集的图像中采集时间相邻的图像。In one embodiment, the first image and the second image are images that are acquired temporally adjacent among the images acquired by the image acquisition module.
在一个实施例中，自动行走装置还包括安装于所述壳体中的传感器，用于在采集图像时同步获取所述自动行走装置的位置参数和/或运动参数；In one embodiment, the automatic running device further includes a sensor installed in the housing, which is used to acquire the position parameters and/or motion parameters of the automatic running device synchronously when collecting images;
所述主控制模块还用于根据所述位置参数和/或运动参数，以及已记录的所述位置变化数据和姿态变化数据计算得到所述自动行走装置的行走轨迹。The main control module is also used to calculate the walking trajectory of the automatic walking device according to the position parameter and/or motion parameter, and the recorded position change data and attitude change data.
在一个实施例中，所述主控制模块还用于检测采集的图像中是否包含预设边界的影像，若是则根据包含预设边界的影像的图像修正所述自动行走装置自前次检测到包含预设边界的影像的图像的时刻至当前时刻的行走轨迹。In one embodiment, the main control module is also used to detect whether the collected image contains an image with a preset boundary, and if so, correct the self-propelled device according to the image containing the preset boundary since the previous detection of the image containing the preset boundary. The walking trajectory from the moment of the image of the boundary image to the current moment.
在一个实施例中，所述主控制模块还用于根据采集的图像检测预设边界，控制所述自动行走装置沿所述预设边界行走一周并记录行走一周的行走轨迹，根据行走一周的行走轨迹确定边界轮廓；或者，In one embodiment, the main control module is also used to detect a preset boundary according to the collected images, control the automatic walking device to walk a circle along the preset boundary and record a walking track for a week, and according to the walking track for a week The trajectory determines the boundary contour; or,
所述主控制模块还用于在人工控制下控制所述自动行走装置沿预设边界行走一周，记录行走一周的行走轨迹，根据行走一周的行走轨迹确定边界轮廓。The main control module is also used to control the automatic walking device to walk one round along the preset boundary under manual control, record the walking track of one round of walking, and determine the boundary contour according to the walking track of one round of walking.
在一个实施例中，所述主控制模块还用于根据所述边界轮廓确定计划行走路径，记录所述自动行走装置的实际行走路径，根据实际行走路径确定所述自动行走装置相对于所述边界轮廓的相对位置，根据所述相对位置控制所述自动行走装置按照所述计划行走路径行走。In one embodiment, the main control module is also used to determine the planned walking path according to the boundary contour, record the actual walking path of the automatic running device, and determine the distance between the automatic running device and the boundary according to the actual walking path. The relative position of the outline, and according to the relative position, the automatic walking device is controlled to walk according to the planned walking path.
一种自动行走装置控制系统，包括上述任一实施例的基于家庭庭院的户外自动行走装置；还包括停靠装置，用于供所述自动行走装置停靠并为所述自动行走装置提供能量。A control system for an automatic running device, comprising the outdoor automatic running device based on a home courtyard in any of the above embodiments; and a docking device for docking the automatic running device and providing energy for the automatic running device.
一种基于家庭庭院的户外自动行走装置控制方法，所述自动行走装置包括：壳体和行走模块，所述行走模块安装于所述壳体上，所述自动行走装置还包括：安装于所述壳体上的图像采集模块；所述方法包括：A control method for an outdoor automatic walking device based on a home garden, the automatic walking device includes: a housing and a walking module, the walking module is installed on the housing, and the automatic walking device also includes: An image acquisition module on the housing; the method includes:
当所述自动行走装置在工作表面上行走时，通过所述图像采集模块连续采集工作表面上的图像；When the automatic walking device walks on the work surface, the images on the work surface are continuously collected by the image acquisition module;
从采集的第一图像中确定特征区域，在所述第一图像后所采集的第二图像中识别所述特征区域；determining a characteristic region from the acquired first image, identifying the characteristic region in a second image acquired subsequent to the first image;
根据所述特征区域的几何变形和位移计算出并记录所述自动行走装置的位置变化数据和姿态变化数据；calculating and recording position change data and attitude change data of the automatic walking device according to the geometric deformation and displacement of the feature area;
根据已记录的所述自动行走装置的位置变化数据和姿态变化数据计算得到所述自动行走装置的行走轨迹。The walking track of the automatic walking device is calculated according to the recorded position change data and attitude change data of the automatic running device.
在一个实施例中，所述自动行走装置还包括安装于所述壳体中的传感器；所述方法还包括：In one embodiment, the self-propelled device further includes a sensor installed in the housing; the method further includes:
在采集图像的同时，通过所述传感器同步地感测所述自动行走装置的位置参数和/或运动参数；synchronously sensing position parameters and/or motion parameters of the automatic walking device through the sensor while collecting images;
所述根据已记录的所述自动行走装置的位置变化数据和姿态变化数据计算得到所述自动行走装置的行走轨迹，包括：The calculation of the walking track of the automatic running device according to the recorded position change data and attitude change data of the automatic running device includes:
根据所述位置参数和/或运动参数，以及已记录的所述位置变化数据和姿态变化数据计算得到所述自动行走装置的行走轨迹。The walking track of the automatic walking device is calculated according to the position parameter and/or motion parameter, and the recorded position change data and attitude change data.
在一个实施例中，所述方法还包括：In one embodiment, the method also includes:
检测采集的图像中是否包含预设边界的影像，若是则根据包含预设边界的影像的图像修正所述自动行走装置自前次检测到包含预设边界的影像的图像的时刻至当前时刻的行走轨迹。Detecting whether the collected image contains the image of the preset boundary, if so, correcting the walking track of the automatic walking device from the time when the image containing the preset boundary was detected last time to the current moment according to the image of the image containing the preset boundary .
在一个实施例中，所述方法还包括：In one embodiment, the method also includes:
根据采集的图像检测预设边界，控制所述自动行走装置沿所述预设边界行走一周并记录行走一周的行走轨迹，根据行走一周的行走轨迹确定边界轮廓；或者，Detecting a preset boundary according to the collected images, controlling the automatic walking device to walk one week along the preset boundary and recording a walking track for one week, and determining a boundary contour according to the walking track for one week; or,
在人工控制下控制所述自动行走装置沿预设边界行走一周，并记录行走一周的行走轨迹，根据行走一周的行走轨迹确定边界轮廓。The automatic walking device is controlled to walk one cycle along the preset boundary under manual control, and the walking track of one week of walking is recorded, and the boundary contour is determined according to the walking track of one week of walking.
在一个实施例中，所述方法还包括：In one embodiment, the method also includes:
根据所述边界轮廓确定计划行走路径；determining a planned walking path according to the boundary contour;
记录所述自动行走装置的实际行走路径；Recording the actual walking path of the automatic walking device;
根据实际行走路径确定所述自动行走装置相对于所述边界轮廓的相对位置；determining the relative position of the automatic walking device relative to the boundary contour according to the actual walking path;
根据所述相对位置控制所述自动行走装置按照所述计划行走路径行走。The automatic walking device is controlled to walk according to the planned walking path according to the relative position.
上述基于家庭庭院的户外自动行走装置、自动行走装置控制系统和方法，通过比对多个图像中特征区域的位移和几何形变从而换算出自动行走装置的位置变化数据和姿态变化数据，进而实现自动行走装置的自定位和得到行走轨迹，方案可靠、成本低，能够准确、有效地控制自动行走装置的行走。The above-mentioned outdoor automatic walking device based on the family courtyard, the automatic walking device control system and method, by comparing the displacement and geometric deformation of the feature areas in multiple images, the position change data and attitude change data of the automatic walking device are converted, and then the automatic walking device is realized. The self-positioning of the walking device and the obtaining of the walking trajectory have a reliable scheme and low cost, and can accurately and effectively control the walking of the automatic walking device.
附图说明Description of drawings
图1为一个实施例中自动行走装置控制系统的结构示意图；Fig. 1 is the structural representation of automatic running device control system in an embodiment;
图2为一个实施例中自动行走装置的简化的结构示意图；Fig. 2 is the simplified structural schematic diagram of self-propelled device in an embodiment;
图3为一个实施例中自动行走装置的简化的结构示意图的局部剖面图；Fig. 3 is the partial sectional view of the simplified structural representation of self-propelled device in an embodiment;
图4为一个实施例中从采集的第一图像中确定特征区域，在所述第一图像后所采集的第二图像中识别所述特征区域的过程的示意图；FIG. 4 is a schematic diagram of a process of determining a feature region from a first image collected and identifying the feature region in a second image collected after the first image in one embodiment;
图5为一个实施例中根据所述特征区域的几何变形和位移计算出所述自动行走装置的位置变化数据和姿态变化数据的过程的示意图；Fig. 5 is a schematic diagram of the process of calculating the position change data and attitude change data of the automatic walking device according to the geometric deformation and displacement of the feature area in one embodiment;
图6为一个实施例中自动行走装置控制方法的流程示意图；Fig. 6 is a schematic flow chart of an automatic running device control method in an embodiment;
图7为另一个实施例中自动行走装置控制方法的流程示意图。Fig. 7 is a schematic flowchart of a method for controlling an automatic traveling device in another embodiment.
具体实施方式detailed description
为了使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本发明进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。In order to make the object, technical solution and advantages of the present invention more clear, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.
如图1所示，在一个实施例中，提供了一种基于家庭庭院的户外自动行走装置控制系统，包括自动行走装置102和供自动行走装置102停靠并为自动行走装置102补充能量的停靠装置104。自动行走装置102行走的表面为其工作表面。在本实施例中，工作表面可以包括工作区域106a和非工作区域106b，工作区域106a和非工作区域106b的交界线形成预设边界106，工作区域106a和非工作区域106b最好有明显区别，预设边界106用于限定自动行走装置102的活动范围。预设边界106具有可辨识性。停靠装置104布置在预设边界106上。自动行走装置102也可以称为机器人。As shown in Fig. 1, in one embodiment, a kind of outdoor self-propelled device control system based on the home yard is provided, including self-propelled device 102 and docking device for self-propelled device 102 and replenishing energy for self-propelled device 102 104. The surface on which the self-propelled device 102 walks is its working surface. In this embodiment, the working surface may include a working area 106a and a non-working area 106b, the boundary line between the working area 106a and the non-working area 106b forms a preset boundary 106, and the working area 106a and the non-working area 106b preferably have a clear distinction, The preset boundary 106 is used to limit the activity range of the automatic walking device 102 . The preset boundary 106 is identifiable. The docking device 104 is arranged on a predetermined boundary 106 . The self-propelled device 102 can also be called a robot.
下面结合图2和图3说明自动行走装置102的具体结构。自动行走装置102具有壳体102a、行走模块、图像采集模块102c、传感器102d和主控制模块（图中未示出）。行走模块和图像采集模块102c安装于壳体102a上，传感器102d和主控制模块安装于壳体102a中。自动行走装置102在行走时，图像采集模块102c和传感器102d相对于壳体102a保持静止状态，且自动行走装置102的行走模块接触工作表面，而壳体102a、图像采集模块102c、传感器102d和主控制模块不接触工作表面。壳体102a起连接固定各个模块的作用，也可以用于保护安装于自动行走装置102的各个模块。主控制模块与行走模块、图像采集模块102c和传感器102d分别连接。The specific structure of the self-propelled device 102 will be described below with reference to FIG. 2 and FIG. 3 . The automatic walking device 102 has a housing 102a, a walking module, an image acquisition module 102c, a sensor 102d and a main control module (not shown in the figure). The walking module and the image acquisition module 102c are installed on the casing 102a, and the sensor 102d and the main control module are installed in the casing 102a. When the automatic traveling device 102 is walking, the image acquisition module 102c and the sensor 102d remain stationary relative to the housing 102a, and the walking module of the automatic traveling device 102 contacts the work surface, while the housing 102a, the image acquisition module 102c, the sensor 102d and the main body The control module does not touch the work surface. The housing 102a is used to connect and fix each module, and can also be used to protect each module installed on the automatic walking device 102 . The main control module is connected with the walking module, the image acquisition module 102c and the sensor 102d respectively.
可以理解的是，自动行走装置102还包括置于壳体102a中的能量模块（图中未示出），能量模块与行走模块、图像采集模块102c、传感器102d和主控制模块分别连接，用于为这些模块提供工作所需的能量。其中，能量模块可以为基于燃料的能量提供模块，燃料可以是汽油或柴油等。能量模块还可以是电池包。在一个实施例中，能量模块为可充电电池包；当自动行走装置102工作时，可充电电池包释放电能以维持自动行走装置102工作；当自动行走装置102不工作时，可充电电池包可以连接到外部电源以补充电能。在一个实施例中，当自动行走装置102检测到可充电电池包电量不足时，自动行走装置102可自动寻找停靠装置104补充电能。It can be understood that the automatic walking device 102 also includes an energy module (not shown in the figure) placed in the housing 102a, and the energy module is respectively connected with the walking module, the image acquisition module 102c, the sensor 102d and the main control module for Provide these modules with the energy they need to work. Wherein, the energy module may be a fuel-based energy supply module, and the fuel may be gasoline or diesel. The energy module can also be a battery pack. In one embodiment, the energy module is a rechargeable battery pack; when the automatic running device 102 is working, the rechargeable battery pack releases electric energy to maintain the automatic running device 102; when the automatic running device 102 is not working, the rechargeable battery pack can Connect to an external power source for supplemental power. In one embodiment, when the automatic running device 102 detects that the rechargeable battery pack is insufficient, the automatic running device 102 can automatically find the docking device 104 to supplement the electric energy.
在其中一个实施例中自动行走装置102还可以包括工作模块，以执行特定的工作。比如工作模块可以是切割模块，包括用于割草的切割部件和驱动切割部件的切割马达，本实施例中自动行走装置102成为割草机。在其它实施例中，还可以为自动行走装置102安装具有其它功能的工作模块，使自动行走装置102成为修剪机、自动吸尘器等。In one of the embodiments, the self-propelled device 102 may also include a working module to perform specific work. For example, the working module may be a cutting module, including a cutting unit for mowing grass and a cutting motor for driving the cutting unit. In this embodiment, the self-propelled device 102 is a lawnmower. In other embodiments, it is also possible to install working modules with other functions for the self-propelled device 102, so that the self-propelled device 102 becomes a trimmer, an automatic vacuum cleaner, and the like.
行走模块用于带动壳体102a及其所安装的各个模块行走，从而实现自动行走装置102的行走。在一个实施例中，行走模块包括轮组102b和用于驱动轮组102b的行走马达（图中未示出）。轮组102b可以有多种设置方法，一般轮组102b的数量为多个，比如至少3个。多个轮组102b，尤其是相对于自动行走装置102的行走方向对称设置的多个轮组102b可以以不同的转速转动，从而使得自动行走装置102可以转向。可以理解的是，行走模块还可以是其它可以实现行走的模块，比如机械臂或履带等，在此不再赘述。The walking module is used to drive the casing 102a and the various modules installed therein to walk, so as to realize the walking of the automatic walking device 102 . In one embodiment, the traveling module includes a wheel set 102b and a traveling motor (not shown in the figure) for driving the wheel set 102b. The wheel set 102b can be arranged in various ways, generally there are multiple wheel sets 102b, such as at least three. The plurality of wheel sets 102b, especially the plurality of wheel sets 102b arranged symmetrically with respect to the traveling direction of the automatic traveling device 102 can rotate at different rotational speeds, so that the automatic traveling device 102 can turn. It can be understood that the walking module can also be other modules that can realize walking, such as a mechanical arm or a crawler, and will not be described in detail here.
图像采集模块102c安装于自动行走装置102上，用于当自动行走装置102在工作表面上行走时，按照预设采样时间间隔连续采集工作表面上的图像。这里的预设采样时间间隔至少要保证采集时间相邻的2个图像有重叠。图像采集模块102c采集图像的范围为图像采集模块102c的采集区域。在一个实施例中，如图3所示，图像采集模块102c位于壳体102a的靠近工作表面的一侧，且其采集方向朝向工作表面，以便于采集自动行走装置102底部、图像采集模块102c下方、特别是正下方的图像。该种方式采集获得的图像畸变小，便于后续处理。图3中300表示该种情况下图像采集模块102c的采集区域的侧视图。在其它实施例中，图像采集模块102c也可以位于壳体102a上任意位置，只需保证图像采集模块102c可以采集到工作表面上的图像即可，比如图像采集模块102c可位于壳体102a上相背于工作表面的一侧或朝向自动行走装置102行走方向的一侧。The image acquisition module 102c is installed on the automatic walking device 102, and is used for continuously collecting images on the working surface according to a preset sampling time interval when the automatic walking device 102 walks on the working surface. The preset sampling time interval here must at least ensure that two images adjacent to each other at the acquisition time overlap. The range of image acquisition by the image acquisition module 102c is the acquisition area of the image acquisition module 102c. In one embodiment, as shown in FIG. 3 , the image acquisition module 102c is located on the side of the housing 102a close to the working surface, and its acquisition direction faces the working surface, so as to collect the bottom of the automatic walking device 102 and below the image acquisition module 102c. , especially the image directly below. The image acquired by this method has little distortion and is convenient for subsequent processing. 300 in FIG. 3 represents a side view of the acquisition area of the image acquisition module 102c in this case. In other embodiments, the image acquisition module 102c can also be located at any position on the housing 102a, it only needs to ensure that the image acquisition module 102c can acquire images on the working surface, for example, the image acquisition module 102c can be located on the housing 102a to The side facing away from the working surface or the side facing the walking direction of the self-propelled device 102 .
在一个实施例中，当图像采集模块102c位于壳体102a的靠近工作表面的一侧，且其采集方向朝向工作表面时，图像采集模块102c与工作表面之间的距离大于或等于3厘米，和/或工作表面上采集区域的大小为大于等于100平方厘米。优选地图像采集模块102c与工作表面之间的距离大于或等于5厘米，和/或工作表面上采集区域的大小为大于等于200平方厘米。这样设置是因为工作表面在微观层面图像变化剧烈，环境不可预知，比如对于割草机来说，草地在微观层面（即几个平方厘米面积的图像）地形相对凹凸不平、图像变化剧烈，环境不可预知，所以采集区域较大些更适于后续处理，以提高对自动行走装置102控制的准确性。In one embodiment, when the image acquisition module 102c is located on the side of the casing 102a close to the work surface, and its acquisition direction faces the work surface, the distance between the image acquisition module 102c and the work surface is greater than or equal to 3 cm, and / or the size of the collection area on the working surface is greater than or equal to 100 square centimeters. Preferably, the distance between the image acquisition module 102c and the working surface is greater than or equal to 5 centimeters, and/or the size of the acquisition area on the working surface is greater than or equal to 200 square centimeters. This setting is because the image of the working surface changes drastically at the micro level, and the environment is unpredictable. It is foreseeable that a larger acquisition area is more suitable for subsequent processing, so as to improve the accuracy of the control of the automatic walking device 102 .
主控制模块用于控制自动行走装置102的行走和自动行走装置102的各个模块的正常工作。主控制模块，与行走模块和图像采集模块102c分别连接，用于连续执行从采集的第一图像中确定特征区域，在第一图像后所采集的第二图像中识别特征区域，根据特征区域的几何变形和位移计算出并记录自动行走装置的位置变化数据和姿态变化数据；主控制模块还用于根据已记录的位置变化数据和姿态变化数据计算得到自动行走装置的行走轨迹。The main control module is used to control the walking of the automatic traveling device 102 and the normal operation of each module of the automatic traveling device 102 . The main control module is connected with the walking module and the image acquisition module 102c respectively, and is used to continuously determine the feature area from the first image collected, identify the feature area in the second image collected after the first image, and identify the feature area according to the feature area. The geometric deformation and displacement calculate and record the position change data and attitude change data of the automatic walking device; the main control module is also used to calculate the walking track of the automatic walking device according to the recorded position change data and attitude change data.
具体地，自动行走装置102在启动后，图像采集模块102c用于按照预设采样时间间隔连续采集工作表面上的图像，并将采集的图像传输给主控制模块。这样主控制模块会收到多个图像。Specifically, after the automatic walking device 102 is started, the image collection module 102c is used to continuously collect images on the working surface according to a preset sampling time interval, and transmit the collected images to the main control module. This way the MCM will receive multiple images.
参照图4，主控制模块用于将采集的其中一个图像402作为第一图像，该第一图像为基准图像，第一图像402后采集的一个图像404作为第二图像，第二图像为对比图像；在第一图像402中选取部分区域作为特征区域402a，在第二图像404中识别出该特征区域402a。接下来主控制模块用于可用于再将图像404作为第一图像，将第一图像404之后采集的一个图像406作为第二图像；在第一图像404中选取部分区域作为特征区域404a，在第二图像406中识别出该特征区域404a。接下来在将图像406作为第一图像时，在第一图像406中选取部分区域作为特征区域406a，这样一直持续执行。当然也可以将采集的一个图像作为第一图像，将该图像之后的另一个图像作为第二图像之后，可以将采集的图像中不同于先前作为第一图像和第二图像的其它图像作为第一图像，再将该图像之后的一个图像作为第二图像。Referring to Fig. 4, the main control module is used to use one of the collected images 402 as a first image, the first image is a reference image, and an image 404 collected after the first image 402 is used as a second image, and the second image is a comparison image Select a partial area in the first image 402 as a feature area 402a, and identify the feature area 402a in the second image 404; Next, the main control module can be used to use the image 404 as the first image, and an image 406 collected after the first image 404 as the second image; select a part of the region in the first image 404 as the feature region 404a, and then The characteristic region 404a is identified in the second image 406 . Next, when the image 406 is used as the first image, a part of the region in the first image 406 is selected as the feature region 406a, and this continues to be performed. Of course, one image collected can also be used as the first image, and another image after the image can be used as the second image, and other images in the collected images that are different from the first image and the second image can be used as the first image. image, and an image following the image is used as the second image.
其中特征区域（402a、404a或406a）可以位于第一图像（402a对应的402、404a对应的404、406a对应的406）的中央位置，这样无论自动行走装置102朝哪个方向移动，都能够尽可能保证第二图像中的特征区域不会缺失。在其它实施例中，特征区域也可以位于第一图像的其它位置，比如在确定自动行走装置102大致的行走方向时，特征区域的选取位置可以稍沿行走方向反向移动一些距离。Wherein the feature area (402a, 404a or 406a) can be located in the central position of the first image (402 corresponding to 402a, 404 corresponding to 404a, 406 corresponding to 406a), so that no matter which direction the automatic walking device 102 moves, it can be as far as possible Guarantees that feature regions in the second image are not missing. In other embodiments, the feature area can also be located at other positions in the first image. For example, when determining the approximate walking direction of the automatic walking device 102 , the selected position of the feature area can be slightly moved in the opposite direction along the walking direction.
主控制模块用于在选取特征区域后，对该特征区域进行处理以提取并记录该特征区域的一些图像特征，提取特征可以采用的方式包括但不限于SIFT（尺度不变特征转换）、SURF（speeded up robust features，快速鲁棒性特征）、DAISY（DAISY是面向稠密特征提取的可快速计算的局部图像特征描述子）、Haar（矩形特征）、WLD（韦伯局部特征）、LBP（LocalBinary Patterns，局部二值模式）、ORB（一种二值特征描述子）、BRIEF（一种特征描述子）、LDA-hash（一种特征描述子）、MSER（Maximally Stable External Regions，区域特征提取）、HOG（Histogram of Oriented Gradient，方向梯度直方图）、灰度值、颜色直方图、灰度直方图、灰度矩等。特征区域可被记录为采集区域的坐标系中的一组位置坐标范围。The main control module is used to process the feature area after selecting the feature area to extract and record some image features of the feature area. The methods that can be used to extract features include but are not limited to SIFT (Scale Invariant Feature Transform), SURF ( speeded up robust features, fast robust features), DAISY (DAISY is a fast-calculating local image feature descriptor for dense feature extraction), Haar (rectangular features), WLD (Weber local features), LBP (LocalBinary Patterns, local binary mode), ORB (a binary feature descriptor), BRIEF (a feature descriptor), LDA-hash (a feature descriptor), MSER (Maximally Stable External Regions, regional feature extraction), HOG (Histogram of Oriented Gradient, directional gradient histogram), gray value, color histogram, gray histogram, gray moment, etc. A feature area may be recorded as a set of position coordinate ranges in the coordinate system of the acquisition area.
在一个实施例中，第一图像和第二图像为图像采集模块102c采集的图像中采集时间相邻的图像。本实施例中，当自动行走装置102在行走时，图像采集模块102c按照预设采样时间间隔连续采集图像，主控制模块将一个图像作为第一图像后，将该图像之后采集的采集时间相邻的下一个图像作为第二图像。这样第一图像和第二图像的共同部分最多，则第二图像中特征区域缺失的可能性最小，位置变化的记录精度也最高。然而在其它实施例中，第一图像和第二图像之间可以存在其它采集的图像，即采集的图像中存在采集时间介于第一图像的采集时间和第二图像的采集时间之间的图像，只需保证第二图像中仍然包含全部或部分的特征区域即可。In one embodiment, the first image and the second image are temporally adjacent images among the images collected by the image collection module 102c. In this embodiment, when the automatic walking device 102 is walking, the image acquisition module 102c continuously acquires images according to the preset sampling time interval. The next image of is used as the second image. In this way, the first image and the second image have the most common parts, the possibility of missing the feature area in the second image is the least, and the recording accuracy of the position change is also the highest. However, in other embodiments, there may be other acquired images between the first image and the second image, that is, there may be images whose acquisition time is between the acquisition time of the first image and the acquisition time of the second image among the acquired images , just ensure that all or part of the feature regions are still contained in the second image.
主控制模块用于在第二图像中识别特征区域，具体地，主控制模块用于根据先前记录的特征区域的图像特征，在第二图像中寻找与记录的图像特征匹配的部分，找到的部分就是识别出的特征区域。The main control module is used to identify the feature area in the second image, specifically, the main control module is used to find the part matching the recorded image feature in the second image according to the image feature of the previously recorded feature area, and the found part is the identified feature region.
以下结合图5来阐述主控制模块如何用于根据特征区域的几何变形和位移计算出自动行走装置102的位置变化数据和姿态变化数据。How the main control module is used to calculate the position change data and posture change data of the automatic walking device 102 according to the geometric deformation and displacement of the feature area will be described below in conjunction with FIG. 5 .
主控制模块用于在第一图像中的特征区域和第二图像中的特征区域确定之后，根据第一图像和第二图像的特征配对结果，计算三维空间中图像采集模块102c在第一图像的采集时间和第二图像的采集时间这两个时刻的变换矩阵，从而推算出图像采集模块102c的位置变化数据和姿态变化数据。由于图像采集模块102c相对于壳体102a是静止的，图像采集模块102c的位置变化数据和姿态变换数据就可以转化为自动行走装置102的位置变化数据和姿态变化数据。其中自动行走装置102的位置变化数据表示自动行走装置102在工作表面上所处的位置。自动行走装置102的姿态变化数据是指表示自动行走装置102因自身的转向和/或地形起伏所导致的自动行走装置102的姿态变化的数据。The main control module is used to calculate, according to the feature pairing results of the first image and the second image after the feature region in the first image and the feature region in the second image are determined, the image acquisition module 102c in the three-dimensional space in the first image The transformation matrix of the acquisition time and the acquisition time of the second image at two moments, so as to calculate the position change data and attitude change data of the image acquisition module 102c. Since the image acquisition module 102c is stationary relative to the casing 102a, the position change data and attitude transformation data of the image acquisition module 102c can be converted into position change data and attitude change data of the automatic walking device 102 . The position change data of the automatic traveling device 102 represents the position of the automatic traveling device 102 on the working surface. The posture change data of the automatic traveling device 102 refers to data representing the posture change of the automatic traveling device 102 caused by its own steering and/or terrain fluctuations.
在一个实施例中，安装于壳体102a中的传感器102d用于在图像采集模块102c采集图像时同步获取自动行走装置102的位置参数和/或运动参数。主控制模块还用于根据位置参数和/或运动参数，以及位置变化数据和姿态变化数据计算得到自动行走装置102的行走轨迹。In one embodiment, the sensor 102d installed in the casing 102a is used to acquire the position parameter and/or the motion parameter of the automatic walking device 102 synchronously when the image acquisition module 102c acquires an image. The main control module is also used to calculate the walking trajectory of the automatic walking device 102 according to the position parameters and/or motion parameters, as well as position change data and attitude change data.
传感器102d包括但不限于角速度传感器、差分GPS（全球定位系统）传感器或磁传感器等。位置参数是指表示自动行走装置102所处位置的信息，比如经纬度；运动参数是指表示自动行走装置102运动状态的信息，比如角速度、加速度、线速度等。主控制模块还用于结合传感器102d所采集的位置参数和/或运动参数，使用滤波算法计算特征区域在第一图像和第二图像中的位移和几何形变。其中滤波算法可以是但不限于：卡尔曼滤波算法、扩展卡尔曼滤波算法、粒子滤波算法等。具体地，主控制模块用于根据第一图像和第二图像中的特征配对结果和传感器采集的位置参数和/或运动参数，计算三维空间中图像采集模块102c在第一图像的采集时间和第二图像的采集时间这两个时刻的变换矩阵，从而推算出图像采集模块102c的位置变化数据和姿态变化数据，进而获得自动行走装置102的位置变化数据和姿态变化数据。The sensor 102d includes, but is not limited to, an angular velocity sensor, a differential GPS (Global Positioning System) sensor, a magnetic sensor, and the like. The position parameter refers to information indicating the location of the automatic walking device 102, such as latitude and longitude; the motion parameter refers to information indicating the motion state of the automatic walking device 102, such as angular velocity, acceleration, and linear velocity. The main control module is further configured to combine the position parameters and/or motion parameters collected by the sensor 102d, and use a filtering algorithm to calculate the displacement and geometric deformation of the feature region in the first image and the second image. The filtering algorithm may be, but not limited to: Kalman filtering algorithm, extended Kalman filtering algorithm, particle filtering algorithm and the like. Specifically, the main control module is used to calculate the acquisition time of the first image and the second acquisition time of the image acquisition module 102c in the three-dimensional space according to the feature pairing results in the first image and the second image and the position parameters and/or motion parameters collected by the sensor. The transformation matrix of the two moments of the acquisition time of the two images is used to calculate the position change data and attitude change data of the image acquisition module 102c, and then obtain the position change data and attitude change data of the automatic walking device 102.
主控制模块用于不断记录自动行走装置102的位置变化数据和姿态变化数据，并根据累积记录的位置变化数据和姿态变化数据计算得到自动行走装置102的行走轨迹。从自动行走装置102启动开始，随着累积的位置变化数据和姿态变化数据的增多，行走轨迹也相应的增长。当然，主控制模块也可以仅记录限定长度的行走轨迹而抹去之前的行走轨迹数据，或者仅记录部分需要的时间段或空间范围内自动行走装置102的行走轨迹，这些可以根据实际工作需要而设定或者调整。The main control module is used to continuously record the position change data and attitude change data of the automatic walking device 102, and calculate the walking trajectory of the automatic walking device 102 according to the accumulated recorded position change data and attitude change data. From the start of the automatic walking device 102, as the accumulated position change data and attitude change data increase, the walking trajectory also increases accordingly. Of course, the main control module can also only record the walking track of a limited length and erase the previous walking track data, or only record the walking track of the automatic walking device 102 in a part of the required time period or space range, which can be adjusted according to actual work needs. set or adjust.
在一个实施例中，主控制模块还用于检测采集的图像中是否包含预设边界106的影像，若是则根据包含预设边界106的影像的图像修正自动行走装置自前次检测到包含预设边界的影像的图像的时刻至当前时刻的行走轨迹。In one embodiment, the main control module is also used to detect whether the collected image contains the image of the preset boundary 106, and if so, correct the automatic walking device according to the image of the image containing the preset boundary 106 since the previous detection contains the preset boundary. The walking trajectory from the moment of the image of the image to the current moment.
本实施例中，在通过图像采集模块102c采集工作表面的图像的同时，同步进行预设边界106的检测。具体地，主控制模块用于实现以下a、b和c的功能：In this embodiment, when the image of the working surface is collected by the image collection module 102c, the detection of the preset boundary 106 is performed synchronously. Specifically, the main control module is used to realize the following functions of a, b and c:
a、将通过图像采集模块102c采集到的图像中的待检测区域使用滑动窗口切分为多个标准尺寸的图像。待检测区域可以是整幅图像，也可以是图像的部分区域，优选为图像中除去中央的特征区域的图像区域。相邻标准尺寸的图像可以有重叠区域，也可以没有重叠区域，视主控制模块的性能而定，优选为有重叠区域。a. Segment the region to be detected in the image collected by the image collection module 102c into multiple images of standard size using a sliding window. The region to be detected may be the entire image, or a partial region of the image, preferably the image region except the central characteristic region in the image. Adjacent standard-sized images may or may not have overlapping areas, depending on the performance of the main control module, preferably overlapping areas.
b、对每个标准尺寸的图像提取特征，送入训练好的分类器中。分类器可以是使用包含预设边界的影像的图像样本和不包含预设边界的影像的图像样本训练获得的分类器。包含预设边界的影像可以是指包含预设边界的局部影像。其中提取的特征可以是人工定义的，也可以是自动训练的，视训练方法而定。其中，人工定义的特征可以是但不限于颜色直方图、LBP、动态纹理、共生矩、HOG、Haar-like（矩形特征）等，优选为颜色直方图和Haar-like特征，因为对实例割草机来说，工作表面一般为草地，草地外的底面为非工作区域。自动训练的特征可使用智能特征选择法或特征学习算法，算法可以是但不限于：Adaboost算法、深度学习中的自编码机、波尔兹曼学习机、卷积神经网络、深置信度网络等。b. Extract features from images of each standard size and send them to the trained classifier. The classifier may be a classifier obtained through training using image samples of images containing preset boundaries and image samples of images not containing preset boundaries. An image containing a predetermined boundary may refer to a partial image containing a predetermined boundary. The extracted features can be manually defined or automatically trained, depending on the training method. Among them, the artificially defined features can be but not limited to color histogram, LBP, dynamic texture, co-occurrence moment, HOG, Haar-like (rectangular feature), etc., preferably color histogram and Haar-like feature, because the instance mowing For machines, the working surface is generally grass, and the bottom surface outside the grass is the non-working area. Automatically trained features can use intelligent feature selection method or feature learning algorithm, the algorithm can be but not limited to: Adaboost algorithm, self-encoder in deep learning, Boltzmann learning machine, convolutional neural network, deep confidence network, etc. .
c、对送入训练好的分类器而输出的分类结果，使用证据理论算法进行融合，最后输出预设边界106的检测结果。c. Fusion the classification results outputted by the trained classifier using the evidence theory algorithm, and finally output the detection result of the preset boundary 106 .
当检测到包含预设边界106的影像的采集图像时，主控制模块用于对自动行走装置102自前次检测到包含预设边界的影像的图像的时刻至当前时刻的行走轨迹进行修正。具体地，主控制模块用于计算得到自动行走装置102的位置与预设边界106的误差，使用误差反向传播算法修正自动行走装置102自前次碰触到预设边界106开始到当前时刻的行走轨迹。相邻两次检测到预设边界的时刻之间的行走轨迹可能不是最佳轨迹，比如自动行走装置102从一点沿曲线运动到另一点，效率低于从一点沿直线到另一点，此时需要对该段轨迹进行修正从而进行优化，以使得自动行走装置102的行走以及工作更加高效。When the captured image of the image containing the preset boundary 106 is detected, the main control module is used to correct the walking trajectory of the automatic walking device 102 from the time when the image containing the preset boundary 106 was detected last time to the current moment. Specifically, the main control module is used to calculate the error between the position of the automatic walking device 102 and the preset boundary 106, and use the error back propagation algorithm to correct the walking of the automatic walking device 102 from the previous time when it touched the preset boundary 106 to the current moment. track. The walking trajectory between two adjacent times when the preset boundary is detected may not be the optimal trajectory. For example, the automatic walking device 102 moves from one point along a curve to another point, and the efficiency is lower than from one point to another point along a straight line. At this time, it is necessary to The trajectory of this section is corrected to optimize, so that the walking and working of the automatic traveling device 102 are more efficient.
在一个实施例中，图像采集模块102c采集图像的预设采样时间间隔可根据自动行走装置102的行走速度决定。当第一次采集图像结束后，在特征区域即将离开采集范围时，需要第二次采集图像，该第一次采集图像和该第二次采集图像的时间间隔为预设采样时间间隔的最大值。In one embodiment, the preset sampling time interval for image collection by the image collection module 102c may be determined according to the walking speed of the automatic walking device 102 . When the first image acquisition ends, when the feature area is about to leave the acquisition range, a second image acquisition is required, and the time interval between the first acquisition image and the second acquisition image is the maximum value of the preset sampling time interval .
在一个实施例中，自动行走装置102具有边界轮廓划定功能，包括自动模式和手动模式。在自动模式下，主控制模块用于根据采集的图像检测预设边界106，控制自动行走装置沿预设边界106行走一周并记录行走一周的行走轨迹，根据行走一周的行走轨迹确定边界轮廓。具体地，图像采集模块102c和传感器102d用于持续采集信号并传送给主控制模块，主控制模块用于根据接收到的信号自动判断自动行走装置102是否移动到预设边界106的边缘，若是则控制自动行走装置102沿预设边界106行走，直至回到起点，沿预设边界106行走一周并记录行走一周的行走轨迹，将行走一周的行走轨迹设置为边界轮廓。在一个实施例中，若自动行走装置102为割草机，则主控制模块用于根据通过图像采集模块102c采集的图像进行边界检测，判断自动行走装置102是否移动到草地边界边缘，若是则控制自动行走装置沿着草地边界行走，直至回到自动行走装置102出发时的起始位置。同时在自动行走装置102行走过程中记录行走轨迹，然后将记录的行走轨迹记录为边界轮廓。In one embodiment, the automatic walking device 102 has a boundary contour delineation function, including an automatic mode and a manual mode. In the automatic mode, the main control module is used to detect the preset boundary 106 according to the collected images, control the automatic walking device to walk one week along the preset boundary 106 and record the walking track of one week of walking, and determine the boundary contour according to the walking track of one week of walking. Specifically, the image acquisition module 102c and the sensor 102d are used to continuously collect signals and transmit them to the main control module, and the main control module is used to automatically judge whether the automatic walking device 102 has moved to the edge of the preset boundary 106 according to the received signals, and if so Control the automatic walking device 102 to walk along the preset boundary 106 until returning to the starting point, walk along the preset boundary 106 for one week and record the walking track for one week, and set the walking track for one week as the boundary contour. In one embodiment, if the self-propelled device 102 is a lawnmower, the main control module is used to perform boundary detection according to the image collected by the image acquisition module 102c, to determine whether the self-propelled device 102 has moved to the border edge of the grassland, and if so, control The automatic running device walks along the border of the grass until getting back to the initial position when the automatic running device 102 started. At the same time, the walking track is recorded during the walking process of the automatic walking device 102, and then the recorded walking track is recorded as a boundary contour.
在手动模式下，主控制模块用于在人工控制下控制自动行走装置沿预设边界106行走一周，记录行走一周的行走轨迹，根据行走一周的行走轨迹确定边界轮廓。具体地，自动行走装置102用于从预设的起点开始，沿预设边界106行走一周，并在行走的过程中持续的记录行走轨迹，然后将记录的行走轨迹记录为边界轮廓。预设的起点可以是停靠装置104，自动行走装置102行走的方式可以是人工带领，如主控制模块用于根据遥控器的遥控信号控制自动行走装置102的行走方向等，也可以为通过人工设置的路标如电子边界线或者指示路标等行走。记录自动行走装置102行走轨迹的方式如自动模式之下的记录方式，不再赘述。In the manual mode, the main control module is used to control the automatic walking device to walk one round along the preset boundary 106 under manual control, record the walking track of one week of walking, and determine the boundary contour according to the walking track of one week of walking. Specifically, the automatic walking device 102 is used to start from a preset starting point, walk along the preset boundary 106 for a week, and continuously record the walking trajectory during the walking process, and then record the recorded walking trajectory as a boundary contour. The preset starting point can be the docking device 104, and the walking mode of the automatic running device 102 can be manually led, such as the main control module is used to control the walking direction of the automatic running device 102 according to the remote control signal of the remote controller, or it can be manually set. Road signs such as electronic boundary lines or directional road signs. The way of recording the walking track of the automatic walking device 102 is like the recording way under the automatic mode, and will not be repeated here.
划定边界轮廓可以为自动行走装置102的工作带来极大的优势，例如，从预设边界106上的已知位置进入工作区域106a后，由于后续的行走轨迹是被持续的记录的，这样自动行走装置102就能够知晓任何时刻它自身的位置以及已经工作过的区域，进而能够保证自动行走装置102不会离开工作区域106a，也可以智能地选择下一步动作，如向哪一个区域行进和工作等。在一个实施例中，主控制模块用于根据边界轮廓划定多个工作区域，并控制自动行走装置102分别在各个工作区域内行走并工作。Delineating the boundary contour can bring great advantages to the work of the automatic walking device 102. For example, after entering the working area 106a from a known position on the preset boundary 106, since the subsequent walking trajectory is continuously recorded, such The self-propelled device 102 can know its own position and the area it has worked at any time, so that it can guarantee that the self-propelled device 102 will not leave the working area 106a, and it can also intelligently select the next action, such as which area to advance and work etc. In one embodiment, the main control module is used to define multiple working areas according to the boundary contour, and control the automatic walking device 102 to walk and work in each working area.
在一个实施例中，主控制模块还用于根据边界轮廓确定计划行走路径，记录自动行走装置的实际行走路径，根据实际行走路径确定自动行走装置相对于边界轮廓的相对位置，根据该相对位置控制自动行走装置按照计划行走路径行走。In one embodiment, the main control module is also used to determine the planned walking path according to the boundary contour, record the actual walking path of the automatic walking device, determine the relative position of the automatic walking device relative to the boundary contour according to the actual walking path, and control the The automatic walking device walks according to the planned walking path.
本实施例中，在自动行走装置102开始工作时，主控制模块用于根据边界轮廓设定一个计划行走路径，该路径是优化的，从而可以避免重复在部分位置工作，按合理的路径行走可节约行走时间。主控制模块用于在设定计划行走路径后控制自动行走装置102行走，并实时记录自动行走装置102的实际行走路径，并根据实际行走路径，或者根据实际行走路径结合位置参数和/或运动参数来计算自动行走装置102相对于边界轮廓的相对位置，该相对位置可以表示当前自动行走装置102在由边界轮廓所形成的边界轮廓地图中的具体位置。然后可根据自动行走装置102的相对位置修正行走方向，以保持自动行走装置102按照计划行走路径行走。在一个实施例中，主控制模块可用于将实际行走路径与计划行走路径进行对比，若不一致则说明此时实际行走路径相对于计划行走路径发生了偏移，主控制模块可用于控制自动行走装置102在行走时反向偏移，以抵消偏移，从而保持自动行走装置102按照计划行走路径行走。In this embodiment, when the automatic walking device 102 starts to work, the main control module is used to set a planned walking path according to the boundary contour. Save walking time. The main control module is used to control the walking of the automatic walking device 102 after setting the planned walking path, and record the actual walking path of the automatic walking device 102 in real time, and according to the actual walking path, or according to the actual walking path combined with position parameters and/or motion parameters To calculate the relative position of the automatic running device 102 relative to the boundary contour, the relative position may represent the specific position of the current automatic running device 102 in the boundary contour map formed by the boundary contour. Then, the walking direction can be corrected according to the relative position of the automatic running device 102, so as to keep the automatic running device 102 walking according to the planned walking path. In one embodiment, the main control module can be used to compare the actual walking path with the planned walking path. If they are inconsistent, it means that the actual walking path has deviated from the planned walking path at this time. The main control module can be used to control the automatic walking device 102 reversely offsets when walking to counteract the offset, thereby keeping the automatic walking device 102 walking according to the planned walking path.
上述自动行走装置102，通过比对多个图像中特征区域的位移和几何形变从而换算出自动行走装置的位置变化数据和姿态变化数据，进而实现自动行走装置的自定位和得到行走轨迹，方案可靠、成本低，能够准确、有效地控制自动行走装置的行走。The above-mentioned automatic walking device 102 converts the position change data and attitude change data of the automatic walking device by comparing the displacement and geometric deformation of the feature regions in multiple images, and then realizes the self-positioning of the automatic walking device and obtains the walking track, and the scheme is reliable. , low cost, and can accurately and effectively control the walking of the automatic walking device.
在一个实施例中，提供了一种基于家庭庭院的户外自动行走装置控制方法，用于控制如上任一实施例的自动行走装置。其中，该自动行走装置包括：壳体、行走模块和图像采集模块；行走模块安装于壳体上，图像采集模块安装于壳体上；如图6所示，该方法包括以下步骤：In one embodiment, a method for controlling an outdoor automatic walking device based on a home courtyard is provided, which is used for controlling the automatic running device in any one of the above embodiments. Wherein, the automatic walking device includes: a housing, a walking module and an image acquisition module; the walking module is installed on the housing, and the image acquisition module is installed on the housing; as shown in Figure 6, the method includes the following steps:
步骤602，当自动行走装置在工作表面上行走时，通过图像采集模块连续采集工作表面上的图像。Step 602, when the automatic walking device walks on the work surface, continuously collect images on the work surface through the image acquisition module.
具体地，当自动行走装置在工作表面上行走时，通过图像采集模块按照预设采样时间间隔连续采集工作表面上的图像。这里的预设采样时间间隔至少要保证采集时间相邻的2个图像有重叠。在一个实施例中，图像采集模块可位于壳体的靠近工作表面的一侧，且其采集方向朝向工作表面，以便于采集自动行走装置底部、图像采集模块下方、特别是正下方的图像。该种方式采集获得的图像畸变小，便于后续处理。在其它实施例中，图像采集模块也可以位于壳体上任意位置，只需保证图像采集模块可以采集到工作表面上的图像即可，比如图像采集模块可位于壳体上相背于工作表面的一侧或朝向自动行走装置行走方向的一侧。Specifically, when the automatic walking device walks on the work surface, the images on the work surface are continuously collected by the image acquisition module according to a preset sampling time interval. The preset sampling time interval here must at least ensure that two images adjacent to each other at the acquisition time overlap. In one embodiment, the image acquisition module can be located on the side of the housing close to the working surface, and its acquisition direction faces the working surface, so as to collect images of the bottom of the automatic walking device, below the image acquisition module, especially directly below. The image acquired by this method has little distortion and is convenient for subsequent processing. In other embodiments, the image acquisition module can also be located at any position on the housing, as long as it is ensured that the image acquisition module can acquire images on the working surface. For example, the image acquisition module can be located on the housing opposite to the working surface. One side or the side facing the walking direction of the automatic walking device.
在一个实施例中，当图像采集模块位于壳体的靠近工作表面的一侧，且其采集方向朝向工作表面时，图像采集模块与工作表面之间的距离大于或等于3厘米，和/或工作表面上采集区域的大小为大于等于100平方厘米。优选地图像采集模块与工作表面之间的距离大于或等于5厘米，和/或工作表面上采集区域的大小为大于等于200平方厘米。这样设置是因为工作表面在微观层面图像变化剧烈，环境不可预知，比如对于割草机来说，草地在微观层面（即几个平方厘米面积的图像）地形相对凹凸不平、图像变化剧烈，环境不可预知，所以采集区域较大些更适于后续处理，以提高对自动行走装置控制的准确性。In one embodiment, when the image acquisition module is located on the side of the housing close to the work surface, and its acquisition direction faces the work surface, the distance between the image acquisition module and the work surface is greater than or equal to 3 cm, and/or the working The size of the collection area on the surface is greater than or equal to 100 square centimeters. Preferably, the distance between the image acquisition module and the working surface is greater than or equal to 5 centimeters, and/or the size of the acquisition area on the working surface is greater than or equal to 200 square centimeters. This setting is because the image of the working surface changes drastically at the micro level, and the environment is unpredictable. Therefore, a larger acquisition area is more suitable for subsequent processing, so as to improve the accuracy of the control of the automatic walking device.
步骤604，从采集的第一图像中确定特征区域，在第一图像后所采集的第二图像中识别特征区域。Step 604, determine the feature area from the first image collected, and identify the feature area in the second image collected after the first image.
具体地，将采集的图像中的其中一个图像作为第一图像，该第一图像为基准图像，第一图像后采集的一个图像作为第二图像，第二图像为对比图像。第一图像中选取部分区域作为特征区域，在第二图像中识别出该特征区域。在后续循环时，可将该第二图像作为后续循环中的第一图像，再将后续循环中的另一个图像作为第二图像。当然也可以将采集的一个图像作为第一图像，将该图像之后的另一个图像作为第二图像之后，可以将采集的图像中不同于先前作为第一图像和第二图像的其它图像作为第一图像，再将该图像之后的一个图像作为第二图像。Specifically, one of the collected images is used as a first image, the first image is a reference image, an image collected after the first image is used as a second image, and the second image is a comparison image. A part of the region is selected in the first image as a characteristic region, and the characteristic region is identified in the second image. In the subsequent cycle, the second image may be used as the first image in the subsequent cycle, and another image in the subsequent cycle may be used as the second image. Of course, one image collected can also be used as the first image, and another image after the image can be used as the second image, and other images in the collected images that are different from the first image and the second image can be used as the first image. image, and an image following the image is used as the second image.
其中特征区域可以位于第一图像的中央位置，这样无论自动行走装置朝哪个方向移动，都能够尽可能保证第二图像中的特征区域不会缺失。在其它实施例中，特征区域也可以位于第一图像的其它位置，比如在确定自动行走装置大致的行走方向时，特征区域的选取位置可以稍沿行走方向反向移动一些距离。The feature area can be located in the center of the first image, so that no matter which direction the automatic walking device moves, it can be ensured that the feature area in the second image will not be missing as much as possible. In other embodiments, the feature area can also be located in other positions of the first image. For example, when determining the approximate walking direction of the automatic walking device, the selected position of the feature area can be slightly moved in the opposite direction along the walking direction.
在提取特征区域之后，需对该特征区域进行处理以提取并记录该特征区域的一些图像特征，提取特征可以采用的方式包括但不限于SIFT、SURF、DAISY、Haar、WLD、LBP、ORB、BRIEF、LDA-hash、MSER、HOG、灰度值、颜色直方图、灰度直方图、灰度矩等。特征区域可被记录为采集区域的坐标系中的一组位置坐标范围。After the feature area is extracted, the feature area needs to be processed to extract and record some image features of the feature area. The methods that can be used to extract features include but are not limited to SIFT, SURF, DAISY, Haar, WLD, LBP, ORB, BRIEF , LDA-hash, MSER, HOG, gray value, color histogram, gray histogram, gray moment, etc. A feature area may be recorded as a set of position coordinate ranges in the coordinate system of the acquisition area.
在一个实施例中，第一图像和第二图像为图像采集模块采集的图像中采集时间相邻的图像。本实施例中，当自动行走装置在行走时，图像采集模块按照预设采样时间间隔连续采集图像，主控制模块将一个图像作为第一图像后，将该图像之后采集的采集时间相邻的下一个图像作为第二图像。这样第一图像和第二图像的共同部分最多，则第二图像中特征区域缺失的可能性最小，位置变化的记录精度也最高。然而在其它实施例中，第一图像和第二图像之间可以存在其它采集的图像，即采集的图像中存在采集时间介于第一图像的采集时间和第二图像的采集时间之间的图像，只需保证第二图像中仍然包含全部或部分的特征区域即可。In one embodiment, the first image and the second image are images that are acquired temporally adjacent among the images acquired by the image acquisition module. In this embodiment, when the automatic walking device is walking, the image acquisition module continuously acquires images according to the preset sampling time interval, and the main control module takes an image as the first image, and then collects the next image with adjacent acquisition time after the image. One image as the second image. In this way, the first image and the second image have the most common parts, the possibility of missing the feature area in the second image is the least, and the recording accuracy of the position change is also the highest. However, in other embodiments, there may be other acquired images between the first image and the second image, that is, there may be images whose acquisition time is between the acquisition time of the first image and the acquisition time of the second image among the acquired images , just ensure that all or part of the feature regions are still contained in the second image.
在第二图像中识别特征区域时，根据先前记录的特征区域的图像特征，在第二图像中寻找与记录的图像特征匹配的部分，找到的部分就是识别出的特征区域。When identifying the feature area in the second image, according to the image feature of the previously recorded feature area, search for a part in the second image that matches the recorded image feature, and the found part is the identified feature area.
步骤606，根据特征区域的几何变形和位移计算出并记录自动行走装置的位置变化数据和姿态变化数据。Step 606, calculate and record the position change data and attitude change data of the automatic walking device according to the geometric deformation and displacement of the feature area.
在第一图像中的特征区域和第二图像中的特征区域确定之后，根据第一图像和第二图像的特征配对结果，计算三维空间中在第一图像的采集时间和第二图像的采集时间这两个时刻的变换矩阵，从而推算出图像采集模块的位置变化数据和姿态变化数据。由于图像采集模块相对于壳体是静止的，图像采集模块的位置变化数据和姿态变换数据就可以转化为自动行走装置的位置变化数据和姿态变化数据。其中自动行走装置的位置变化数据表示自动行走装置在工作表面上所处的位置。自动行走装置的姿态变化数据是指表示自动行走装置因自身的转向和/或地形起伏所导致的自动行走装置的姿态变化的数据。After the feature area in the first image and the feature area in the second image are determined, according to the feature pairing results of the first image and the second image, calculate the acquisition time of the first image and the acquisition time of the second image in three-dimensional space The transformation matrix at these two moments can be used to calculate the position change data and attitude change data of the image acquisition module. Since the image acquisition module is stationary relative to the casing, the position change data and attitude transformation data of the image acquisition module can be converted into position change data and attitude change data of the automatic walking device. Wherein the position change data of the automatic traveling device indicates the position of the automatic traveling device on the working surface. The posture change data of the automatic traveling device refers to the data representing the posture change of the automatic traveling device caused by its own steering and/or terrain fluctuations.
步骤608，根据已记录的自动行走装置的位置变化数据和姿态变化数据计算得到自动行走装置的行走轨迹。Step 608, calculating the walking trajectory of the automatic walking device according to the recorded position change data and attitude change data of the automatic walking device.
不断记录自动行走装置的位置变化数据和姿态变化数据，并根据累积记录的位置变化数据和姿态变化数据计算得到自动行走装置的行走轨迹。从自动行走装置启动开始，随着累积的位置变化数据和姿态变化数据的增多，行走轨迹也相应的增长。当然，也可以仅记录限定长度的行走轨迹而抹去之前的行走轨迹数据，或者仅记录部分需要的时间段或空间范围内自动行走装置的行走轨迹，这些可以根据实际工作需要而设定或者调整。Continuously record the position change data and attitude change data of the automatic walking device, and calculate the walking trajectory of the automatic walking device according to the accumulated recorded position change data and attitude change data. From the start of the automatic walking device, as the accumulated position change data and attitude change data increase, the walking trajectory also increases accordingly. Of course, it is also possible to only record the walking track of a limited length and erase the previous walking track data, or only record the walking track of the automatic walking device in a part of the required time period or space range, which can be set or adjusted according to actual work needs .
上述自动行走装置控制方法，通过比对多个图像中特征区域的位移和几何形变从而换算出自动行走装置的位置变化数据和姿态变化数据，进而实现自动行走装置的自定位和得到行走轨迹，方案可靠、成本低，能够准确、有效地控制自动行走装置的行走。The control method of the above-mentioned automatic walking device converts the position change data and attitude change data of the automatic walking device by comparing the displacement and geometric deformation of the feature areas in multiple images, and then realizes the self-positioning of the automatic walking device and obtains the walking trajectory. The utility model has the advantages of reliability and low cost, and can accurately and effectively control the walking of the automatic traveling device.
在一个实施例中，自动行走装置还包括安装于壳体中的传感器。该自动行走装置控制方法还包括：在采集图像的同时，通过传感器同步地感测自动行走装置的位置参数和/或运动参数。则步骤608包括：根据位置参数和/或运动参数，以及位置变化数据和姿态变化数据计算得到自动行走装置的行走轨迹。In one embodiment, the self-propelled device further includes a sensor installed in the casing. The automatic running device control method further includes: synchronously sensing position parameters and/or motion parameters of the automatic running device through sensors while collecting images. Then step 608 includes: calculating the walking track of the automatic walking device according to the position parameter and/or motion parameter, as well as position change data and attitude change data.
传感器包括但不限于角速度传感器、差分GPS传感器或磁传感器等。结合传感器所采集的位置参数和/或运动参数，使用滤波算法计算特征区域在第一图像和第二图像中的位移和几何形变。其中滤波算法可以是但不限于：卡尔曼滤波算法、扩展卡尔曼滤波算法、粒子滤波算法等。具体地，可根据第一图像和第二图像中的特征配对结果和传感器采集的位置参数和/或运动参数，计算三维空间中图像采集模块在第一图像的采集时间和第二图像的采集时间这两个时刻的变换矩阵，从而推算出图像采集模块的位置变化数据和姿态变化数据，进而获得自动行走装置的位置变化数据和姿态变化数据。Sensors include but are not limited to angular velocity sensors, differential GPS sensors, or magnetic sensors. Combined with the position parameter and/or motion parameter collected by the sensor, a filtering algorithm is used to calculate the displacement and geometric deformation of the feature region in the first image and the second image. The filtering algorithm may be, but not limited to: Kalman filtering algorithm, extended Kalman filtering algorithm, particle filtering algorithm and the like. Specifically, the acquisition time of the first image and the acquisition time of the second image by the image acquisition module in three-dimensional space can be calculated according to the feature pairing results in the first image and the second image and the position parameters and/or motion parameters collected by the sensor The transformation matrix at these two moments is used to calculate the position change data and attitude change data of the image acquisition module, and then obtain the position change data and attitude change data of the automatic walking device.
在一个实施例中，该自动行走装置控制方法还包括：检测采集的图像中是否包含预设边界的影像，若是则根据包含预设边界的影像的图像修正自动行走装置自前次检测到包含预设边界的影像的图像的时刻至当前时刻的行走轨迹。In one embodiment, the automatic running device control method further includes: detecting whether the collected image contains an image with a preset boundary, and if so, correcting the automatic running device according to the image containing the preset boundary since the previous detection of the image containing the preset boundary. The walking trajectory from the moment of the image of the boundary image to the current moment.
本实施例中，在通过图像采集模块c采集工作表面的图像的同时，同步进行预设边界的检测，具体地，检测预设边界的步骤包括步骤1）～步骤3）：In this embodiment, while the image of the working surface is collected by the image acquisition module c, the detection of the preset boundary is performed synchronously. Specifically, the steps of detecting the preset boundary include step 1) to step 3):
步骤1），将通过图像采集模块采集到的图像中的待检测区域使用滑动窗口切分为多个标准尺寸的图像。待检测区域可以是整幅图像，也可以是图像的部分区域，优选为图像中除去中央的特征区域的图像区域。相邻标准尺寸的图像可以有重叠区域，也可以没有重叠区域，优选为有重叠区域。Step 1), the area to be detected in the image collected by the image acquisition module is divided into multiple images of standard size using a sliding window. The region to be detected may be the entire image, or a partial region of the image, preferably the image region except the central characteristic region in the image. Adjacent images of standard sizes may or may not have overlapping areas, but preferably have overlapping areas.
步骤2），对每个标准尺寸的图像提取特征，送入训练好的分类器中。分类器可以是使用包含预设边界的影像的图像样本和不包含预设边界的影像的图像样本训练获得的分类器。包含预设边界的影像可以是指包含预设边界的局部影像。其中提取的特征可以是人工定义的，也可以是自动训练的，视训练方法而定。其中，人工定义的特征可以是但不限于颜色直方图、LBP、动态纹理、共生矩、HOG、Haar-like等，优选为颜色直方图和Haar-like特征，因为对实例割草机来说，工作表面一般为草地，草地外的底面为非工作区域。自动训练的特征可使用智能特征选择法或特征学习算法，算法可以是但不限于：Adaboost算法、深度学习中的自编码机、波尔兹曼学习机、卷积神经网络、深置信度网络等。Step 2), extract features for each standard size image, and send it to the trained classifier. The classifier may be a classifier obtained through training using image samples of images containing preset boundaries and image samples of images not containing preset boundaries. An image containing a predetermined boundary may refer to a partial image containing a predetermined boundary. The extracted features can be manually defined or automatically trained, depending on the training method. Among them, the artificially defined features can be but not limited to color histogram, LBP, dynamic texture, co-occurrence moment, HOG, Haar-like, etc., preferably color histogram and Haar-like features, because for the example lawn mower, The working surface is generally grass, and the bottom surface outside the grass is the non-working area. Automatically trained features can use intelligent feature selection method or feature learning algorithm, the algorithm can be but not limited to: Adaboost algorithm, self-encoder in deep learning, Boltzmann learning machine, convolutional neural network, deep confidence network, etc. .
步骤3），对送入训练好的分类器而输出的分类结果，使用证据理论算法进行融合，最后输出预设边界的检测结果。Step 3), use the evidence theory algorithm to fuse the classification results output by the trained classifier, and finally output the detection results of the preset boundaries.
当检测到包含预设边界的影像的采集图像时，对自动行走装置自前次检测到包含预设边界的影像的图像的时刻至当前时刻的行走轨迹进行修正。具体地，计算得到自动行走装置的位置与预设边界的误差，使用误差反向传播算法修正自动行走装置自前次碰触到预设边界开始到当前时刻的行走轨迹。相邻两次检测到预设边界的时刻之间的行走轨迹可能不是最佳轨迹，比如自动行走装置从一点沿曲线运动到另一点，效率低于从一点沿直线到另一点，此时需要对该段轨迹进行修正从而进行优化，以使得自动行走装置的行走以及工作更加高效。When the captured image of the image containing the preset boundary is detected, the walking track of the automatic walking device is corrected from the time when the image containing the preset boundary was detected last time to the current moment. Specifically, the error between the position of the automatic walking device and the preset boundary is calculated, and the error back propagation algorithm is used to correct the walking trajectory of the automatic walking device from the previous time when it touched the preset boundary to the current moment. The walking trajectory between two adjacent times when the preset boundary is detected may not be the best trajectory. For example, the automatic walking device moves from one point along a curve to another point, and the efficiency is lower than from one point to another point along a straight line. At this time, it is necessary to This section of trajectory is corrected and optimized to make the walking and working of the automatic walking device more efficient.
在一个实施例中，图像采集模块采集图像的预设采样时间间隔可根据自动行走装置的行走速度决定。当第一次采集图像结束后，在特征区域即将离开采集范围时，需要第二次采集图像，该第一次采集图像和该第二次采集图像的时间间隔为预设采样时间间隔的最大值。In one embodiment, the preset sampling time interval for image acquisition by the image acquisition module may be determined according to the walking speed of the automatic walking device. When the first image acquisition ends, when the feature area is about to leave the acquisition range, a second image acquisition is required, and the time interval between the first acquisition image and the second acquisition image is the maximum value of the preset sampling time interval .
在一个实施例中，该自动行走装置控制方法还包括步骤：根据采集的图像检测预设边界，控制自动行走装置沿预设边界行走一周并记录行走一周的行走轨迹，根据行走一周的行走轨迹确定边界轮廓。In one embodiment, the automatic walking device control method further includes the step of: detecting a preset boundary according to the collected images, controlling the automatic walking device to walk one week along the preset boundary and recording the walking trajectory of one week of walking, and determining Boundary outline.
自动行走装置具有边界轮廓划定功能，包括自动模式和手动模式。在自动模式下，通过图像采集模块和传感器持续采集信号，并根据采集到的信号自动判断自动行走装置是否移动到预设边界的边缘，若是则控制自动行走装置沿预设边界行走，直至回到起点，沿预设边界行走一周并记录行走一周的行走轨迹，将行走一周的行走轨迹设置为边界轮廓。在一个实施例中，若自动行走装置为割草机，则可根据通过图像采集模块采集的图像进行边界检测，判断自动行走装置是否移动到草地边界边缘，若是则控制自动行走装置沿着草地边界行走，直至回到自动行走装置出发时的起始位置。同时在自动行走装置行走过程中记录行走轨迹，然后将记录的行走轨迹记录为边界轮廓。The automatic walking device has the function of boundary contour delineation, including automatic mode and manual mode. In the automatic mode, the image acquisition module and the sensor continue to collect signals, and automatically judge whether the automatic walking device has moved to the edge of the preset boundary according to the collected signals, and if so, control the automatic walking device to walk along the preset boundary until it returns to the edge of the preset boundary. The starting point is to walk for one week along the preset boundary and record the walking track for one week, and set the walking track for one week as the boundary contour. In one embodiment, if the automatic walking device is a lawn mower, boundary detection can be carried out according to the image collected by the image acquisition module to determine whether the automatic walking device has moved to the border edge of the grassland, and if so, control the automatic walking device along the border of the grassland Walk until getting back to the starting position when the automatic running device starts. At the same time, the walking track is recorded during the walking process of the automatic walking device, and then the recorded walking track is recorded as a boundary contour.
在一个实施例中，该自动行走装置控制方法还包括步骤：在人工控制下控制自动行走装置沿预设边界行走一周，并记录行走一周的行走轨迹，根据行走一周的行走轨迹确定边界轮廓。In one embodiment, the automatic running device control method further includes the step of: controlling the automatic running device to walk one round along the preset boundary under manual control, recording the walking track of one round of walking, and determining the boundary contour according to the walking track of one round of walking.
在手动模式下，在人工控制下控制自动行走装置沿预设边界行走一周，记录行走一周的行走轨迹，根据行走一周的行走轨迹确定边界轮廓。具体地，自动行走装置从预设的起点开始，沿预设边界行走一周，并在行走的过程中持续的记录行走轨迹，然后将记录的行走轨迹记录为边界轮廓。预设的起点可以是停靠装置，自动行走装置行走的方式可以是人工带领，如自动行走装置可在遥控信号的控制下控制其行走方向等，也可以为通过人工设置的路标如电子边界线或者指示路标等行走。记录自动行走装置行走轨迹的方式如自动模式之下的记录方式，不再赘述。In the manual mode, the automatic walking device is controlled to walk a circle along the preset boundary under manual control, the walking track of the walking circle is recorded, and the boundary contour is determined according to the walking track of the walking circle. Specifically, the automatic walking device starts from a preset starting point, walks along a preset boundary for one week, and continuously records the walking trajectory during the walking process, and then records the recorded walking trajectory as a boundary contour. The preset starting point can be a docking device, and the walking mode of the automatic traveling device can be manually led, such as the automatic traveling device can control its walking direction under the control of a remote control signal, or it can be a manually set road sign such as an electronic boundary line or Instruct road signs etc. to walk. The way of recording the walking track of the automatic walking device is like the recording method under the automatic mode, and will not be described in detail.
划定边界轮廓可以为自动行走装置的工作带来极大的优势，例如，从预设边界上的已知位置进入工作区域后，由于后续的行走轨迹是被持续的记录的，这样自动行走装置就能够知晓任何时刻它自身的位置以及已经工作过的区域，进而能够保证自动行走装置不会离开工作区域，也可以智能地选择下一步动作，如向哪一个区域行进和工作等。在一个实施例中，还可以根据边界轮廓划定多个工作区域，并控制自动行走装置分别在各个工作区域内行走并工作。Delineating the boundary contour can bring great advantages to the work of the automatic walking device. For example, after entering the work area from a known position on the preset boundary, since the subsequent walking trajectory is continuously recorded, the automatic walking device It can know its own position and the area it has worked at any time, so as to ensure that the automatic walking device will not leave the working area, and can also intelligently choose the next action, such as which area to go and work. In one embodiment, a plurality of working areas can also be defined according to the boundary contour, and the automatic walking device can be controlled to walk and work in each working area respectively.
在一个实施例中，该自动行走装置控制方法还包括：根据边界轮廓确定计划行走路径；记录自动行走装置的实际行走路径；根据实际行走路径确定自动行走装置相对于边界轮廓的相对位置；根据该相对位置控制自动行走装置按照计划行走路径行走。In one embodiment, the automatic running device control method also includes: determining the planned walking path according to the boundary contour; recording the actual walking path of the automatic running device; determining the relative position of the automatic running device with respect to the boundary contour according to the actual walking path; The relative position controls the automatic walking device to walk according to the planned walking path.
本实施例中，在自动行走装置开始工作时，根据边界轮廓设定一个计划行走路径，该路径是优化的，从而可以避免重复在部分位置工作，按合理的路径行走可节约行走时间。在设定计划行走路径后控制自动行走装置行走，并实时记录自动行走装置的实际行走路径，并根据实际行走路径计算自动行走装置相对于边界轮廓的相对位置，并根据自动行走装置的相对位置修正行走方向，以保持自动行走装置按照计划行走路径行走。In this embodiment, when the automatic walking device starts to work, a planned walking path is set according to the boundary contour. The path is optimized, thereby avoiding repeated work in some positions, and walking according to a reasonable path can save walking time. After setting the planned walking path, control the walking of the automatic walking device, and record the actual walking path of the automatic walking device in real time, and calculate the relative position of the automatic walking device relative to the boundary contour according to the actual walking path, and correct it according to the relative position of the automatic walking device Walking direction to keep the automatic walking device walking according to the planned walking path.
参考图7，在另一个实施例中，提供了一种自动行走装置控制方法，包括以下步骤：Referring to Fig. 7, in another embodiment, a kind of automatic running device control method is provided, comprising the following steps:
步骤702，当自动行走装置在工作表面上行走时，通过图像采集模块连续采集工作表面上的图像。Step 702, when the automatic walking device walks on the work surface, continuously collect images on the work surface through the image acquisition module.
在执行步骤702时执行步骤703，在采集图像的同时，通过传感器同步地感测自动行走装置的位置参数和/或运动参数。Step 703 is executed when step 702 is executed, and the position parameter and/or motion parameter of the automatic walking device is sensed synchronously through the sensor while the image is being collected.
在步骤702之后，执行步骤705，检测边界的步骤。具体包括：检测采集的图像中是否包含预设边界的影像，若是则执行步骤710。After step 702, execute step 705, a step of detecting boundaries. Specifically, it includes: detecting whether the collected image contains an image of a preset boundary, and if so, performing step 710 .
步骤704，从采集的第一图像中确定特征区域，在第一图像后所采集的第二图像中识别特征区域。Step 704, determine the feature area from the first image collected, and identify the feature area in the second image collected after the first image.
步骤706，根据特征区域的几何变形和位移计算出并记录自动行走装置的位置变化数据和姿态变化数据。Step 706, calculate and record the position change data and attitude change data of the automatic walking device according to the geometric deformation and displacement of the feature area.
步骤708，根据位置参数和/或运动参数，以及已记录的位置变化数据和姿态变化数据计算得到自动行走装置的行走轨迹。Step 708, calculate the walking trajectory of the automatic walking device according to the position parameters and/or motion parameters, as well as the recorded position change data and attitude change data.
步骤710，当检测到包含预设边界的影像的采集图像时，对自动行走装置自前次检测到包含预设边界的影像的图像的时刻至当前时刻的行走轨迹进行修正。Step 710, when the captured image of the image containing the preset boundary is detected, correct the walking trajectory of the automatic walking device from the time when the image containing the preset boundary was detected last time to the current moment.
上述步骤702～步骤710循环执行。The above steps 702 to 710 are executed cyclically.
以上所述实施例仅表达了本发明的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对本发明专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本发明构思的前提下，还可以做出若干变形和改进，这些都属于本发明的保护范围。因此，本发明专利的保护范围应以所附权利要求为准。The above-mentioned embodiments only express several implementation modes of the present invention, and the descriptions thereof are relatively specific and detailed, but should not be construed as limiting the patent scope of the present invention. It should be pointed out that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent for the present invention should be based on the appended claims.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410140724.6ACN103901890B (en) | 2014-04-09 | 2014-04-09 | Outdoor automatic walking device based on family courtyard and system and method for controlling outdoor automatic walking device based on family courtyard |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410140724.6ACN103901890B (en) | 2014-04-09 | 2014-04-09 | Outdoor automatic walking device based on family courtyard and system and method for controlling outdoor automatic walking device based on family courtyard |
| Publication Number | Publication Date |
|---|---|
| CN103901890A CN103901890A (en) | 2014-07-02 |
| CN103901890Btrue CN103901890B (en) | 2017-05-24 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410140724.6AActiveCN103901890B (en) | 2014-04-09 | 2014-04-09 | Outdoor automatic walking device based on family courtyard and system and method for controlling outdoor automatic walking device based on family courtyard |
| Country | Link |
|---|---|
| CN (1) | CN103901890B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105512689A (en)* | 2014-09-23 | 2016-04-20 | 苏州宝时得电动工具有限公司 | Lawn identification method based on images, and lawn maintenance robot |
| US9516806B2 (en) | 2014-10-10 | 2016-12-13 | Irobot Corporation | Robotic lawn mowing boundary determination |
| WO2017206950A1 (en)* | 2016-06-03 | 2017-12-07 | 苏州宝时得电动工具有限公司 | Automatic walking device and method for controlling the walking thereof |
| CN107463168A (en)* | 2016-06-06 | 2017-12-12 | 苏州宝时得电动工具有限公司 | Localization method and system, map constructing method and system, automatic running device |
| IT201700058505A1 (en) | 2017-05-30 | 2018-11-30 | Volta Robots S R L | Method of control of a soil processing vehicle based on image processing and related system |
| CN107306923A (en)* | 2017-06-05 | 2017-11-03 | 广西英伟科技有限公司 | It is a kind of can automatically walk pesticide spraying device |
| TWI673586B (en)* | 2018-02-14 | 2019-10-01 | 國立交通大學 | Online training enabled trail following system and method |
| CN111198557B (en)* | 2018-10-31 | 2022-03-18 | 苏州科瓴精密机械科技有限公司 | Control method and control system for walking robot |
| CN109859158A (en)* | 2018-11-27 | 2019-06-07 | 邦鼓思电子科技（上海）有限公司 | A kind of detection system, method and the machinery equipment on the working region boundary of view-based access control model |
| CN110109454A (en)* | 2019-04-18 | 2019-08-09 | 福建信息职业技术学院 | A kind of intelligent carriage motion profile clone method |
| WO2021139414A1 (en)* | 2020-01-06 | 2021-07-15 | 苏州宝时得电动工具有限公司 | Self-moving device |
| CN111328017B (en)* | 2020-02-18 | 2021-05-14 | 深圳市愚公科技有限公司 | Map transmission method and device |
| CN113885485B (en)* | 2020-06-17 | 2023-12-22 | 苏州科瓴精密机械科技有限公司 | Robot walking control method, system, robot and storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101727756A (en)* | 2008-10-16 | 2010-06-09 | 财团法人工业技术研究院 | Mobile image aided guidance method and system for vehicle |
| CN102662400A (en)* | 2012-05-10 | 2012-09-12 | 慈溪思达电子科技有限公司 | Path planning algorithm of mowing robot |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4954962A (en)* | 1988-09-06 | 1990-09-04 | Transitions Research Corporation | Visual navigation and obstacle avoidance structured light system |
| US7881497B2 (en)* | 2007-03-08 | 2011-02-01 | Honeywell International Inc. | Vision based navigation and guidance system |
| CN102545275A (en)* | 2010-12-07 | 2012-07-04 | 上海新世纪机器人有限公司 | Robot automatic charging device and robot automatic charging method |
| CN102681541A (en)* | 2011-03-10 | 2012-09-19 | 上海方伴自动化设备有限公司 | Method for image recognition and vision positioning with robot |
| CN103324191A (en)* | 2012-03-23 | 2013-09-25 | 苏州宝时得电动工具有限公司 | Control method and control system executing same |
| CN103324192A (en)* | 2012-03-23 | 2013-09-25 | 苏州宝时得电动工具有限公司 | Boundary setting method and boundary setting system |
| CN103586870B (en)* | 2013-11-05 | 2016-06-29 | 中国科学院深圳先进技术研究院 | Steam generator climbing robot |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101727756A (en)* | 2008-10-16 | 2010-06-09 | 财团法人工业技术研究院 | Mobile image aided guidance method and system for vehicle |
| CN102662400A (en)* | 2012-05-10 | 2012-09-12 | 慈溪思达电子科技有限公司 | Path planning algorithm of mowing robot |
| Publication number | Publication date |
|---|---|
| CN103901890A (en) | 2014-07-02 |
| Publication | Publication Date | Title |
|---|---|---|
| CN103901890B (en) | Outdoor automatic walking device based on family courtyard and system and method for controlling outdoor automatic walking device based on family courtyard | |
| KR102314539B1 (en) | Controlling method for Artificial intelligence Moving robot | |
| CN109730590B (en) | Cleaning robot and method for automatically returning and charging same | |
| CN110477825B (en) | Cleaning robot, autonomous charging method, system, and readable storage medium | |
| CN111035327B (en) | Cleaning robot, carpet detection method and computer-readable storage medium | |
| CN114847803B (en) | Robot positioning method and device, electronic device, and storage medium | |
| AU2017316089B2 (en) | Mobile robot and control method therefor | |
| US10102429B2 (en) | Systems and methods for capturing images and annotating the captured images with information | |
| CN112184736B (en) | Multi-plane extraction method based on European clustering | |
| CN110522359A (en) | Cleaning robot and control method of cleaning robot | |
| US9037396B2 (en) | Simultaneous localization and mapping for a mobile robot | |
| JP2022540387A (en) | Mobile robot and its control method | |
| CN114603559A (en) | Control method and device for mobile robot, mobile robot and storage medium | |
| WO2020113452A1 (en) | Monitoring method and device for moving target, monitoring system, and mobile robot | |
| CN107239074A (en) | Automatic working system and its map method for building up of working region | |
| WO2014173290A1 (en) | Automatic walking device and method for determining working area thereof | |
| CN207051978U (en) | A kind of system for reaching indoor task object position determination by image recognition mode | |
| CN112075879A (en) | Information processing method, device and storage medium | |
| CN108415419A (en) | Rubbish pick-up method, robot, electronic equipment and computer program product | |
| CN114683290B (en) | Method and device for optimizing pose of foot robot and storage medium | |
| CN104850120B (en) | Wheeled mobile robot air navigation aid based on IHDR autonomous learning frameworks | |
| CN114034299A (en) | Navigation system based on active laser SLAM | |
| CN205880664U (en) | Intelligence weeder | |
| CN113885495A (en) | An outdoor automatic work control system, method and device based on machine vision | |
| CN112417944A (en) | A robot control method and electronic device |
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |