Disclosure of Invention
Aiming at the defects in the prior art, the intelligent fish monitoring method based on the video images solves the problem that the number of each type of fish cannot be counted based on the Hu invariant moment in the prior art.
In order to achieve the purpose of the invention, the invention adopts the technical scheme that:
s1, obtaining a fish passing video at a t-time period at the entrance of the fishway, and performing frame extraction processing on the fish passing video to obtain a plurality of frame images;
s2, performing noise reduction processing on all frame images, and then sequentially inputting the frame images into the trained fast-RCNN according to the time sequence to obtain the identification information of each fish in the frame images; the identification information comprises the length and width of a frame where the fish is located, the center position and the probability of belonging to each type of fish;
s3, calculating the distance between the center position of each fish in the frame image i and the center positions of all the same type of fish in the frame image i +1 according to the identification information of each fish in the two adjacent frame images i, i + 1;
s4, judging whether one distance between the fish j in the frame image i and the same type of fish in the frame image i +1 is smaller than a preset distance; if yes, go to step S5, otherwise go to step S7;
s5, enabling the fish j to exist in the frame images i, i +1 at the same time, then judging whether the central positions of the fish j on the frame images i, i +1 are respectively positioned at two sides of the reference line, if so, entering a step S6, otherwise, entering a step S7;
s6, judging whether the variation of the center position of the fish j in the frame image i, i +1 is a positive value, if so, adding 1 to the number of the fish of the type of the fish j, and entering the step S7, otherwise, subtracting 1 from the number of the fish of the type of the fish j, and entering the step S7;
s7, judging whether all the fishes in the frame image i are judged to be finished, if yes, entering a step S8, otherwise, enabling j to be j +1, and returning to the step S4;
s8, determining whether all the frame images extracted in the time period t have been determined, if yes, making t equal to t +1, and returning to step S1, otherwise, making i equal to i +1, and returning to step S3.
The invention has the beneficial effects that: according to the scheme, firstly, the fish type in each frame of image is identified through a neural network, so that the problem that the fish type is inaccurately determined due to the fact that the fish posture is not standard during grating counting in the prior art is solved; the same fish is determined based on the central positions of two adjacent fishes of the same type, and the same fish in the two adjacent frame images can be accurately determined, so that the accuracy of subsequent counting of the same type of fish is ensured.
The movement direction of the fish can be accurately determined through the variation of the center position of the same fish, so that whether the fish at the entrance of the fishway enters or swims out of the fishway is determined, and accurate calculation of each type of fish is guaranteed.
Through the number of each type of fish passing through the fishway, managers can be assisted to know the applicability of each type to the fishway, so that the managers are promoted to improve the fishway, and the fishway is applicable to all types of fish in the basin as much as possible.
Detailed Description
The following description of the embodiments of the present invention is provided to facilitate the understanding of the present invention by those skilled in the art, but it should be understood that the present invention is not limited to the scope of the embodiments, and it will be apparent to those skilled in the art that various changes may be made without departing from the spirit and scope of the invention as defined and defined in the appended claims, and all matters produced by the invention using the inventive concept are protected.
Referring to fig. 1, fig. 1 illustrates a fish intelligent monitoring method based on video images; as shown in fig. 1, the method S includes steps S1 to S8.
In step S1, obtaining a fish passing video at a t-time period at the entrance of the fishway, and performing frame extraction on the fish passing video to obtain a plurality of frame images; in order to ensure the accuracy of detection, the scheme preferably extracts 30 frames of images per second of video.
The fish video shooting angle of crossing in this scheme is perpendicular with fishway entry axis, through the setting of shooting camera angle, can avoid shooting the improper fish image appearance deformity in the angle arouses the image, and influences the accuracy of follow-up fish species discernment.
In step S2, performing noise reduction processing on all frame images, and then sequentially inputting the frame images into the trained fast-RCNN in time order to obtain identification information of each fish in the frame images; the identification information comprises the length and width of a frame where the fish is located, the center position and the probability of belonging to each type of fish.
In the scheme, when the noise reduction processing is performed, a Gaussian low-pass filter is preferably adopted to perform filtering processing on the frame image. In order to verify the filtering effect of the Gaussian low-pass filter, the method is compared and analyzed with a common filter in the prior art:
25 samples are obtained, filtering processing is carried out on the 25 samples by adopting an ideal low-pass filter, a Butterworth low-pass filter, a Gaussian low-pass filter, an ideal high-pass filter, a Butterworth low-pass filter and a Gaussian high-pass filter respectively, comparison results of processing results of the filters and non-processing results are shown in figure 2, and the accuracy rate of the 25 samples after processing is shown in table 1.
TABLE 1 accuracy of filter processing of 25 samples with several filters
As can be seen from table 1, the gaussian low-pass filter employed in the present application has the best filtering effect.
In step S3, calculating the distance between the center position of each fish in the frame image i and the center positions of all the same type of fish in the frame image i +1 according to the identification information of each fish in the two adjacent frame images i, i + 1;
in step S4, it is determined whether there is a distance smaller than a preset distance between the fish j in the frame image i and the same type of fish in the frame image i + 1; if yes, go to step S5, otherwise go to step S7;
in step S5, the fish j is present in the frame images i, i +1 at the same time, and then it is determined whether the center positions of the fish j on the frame images i, i +1 are located on both sides of the reference line, if yes, step S6 is performed, otherwise, step S7 is performed;
in step S6, it is determined whether the variation of the center position of the fish j in the frame image i, i +1 is a positive value, if yes, the number of the type fish of the fish j is increased by 1, and step S7 is performed, otherwise, the number of the type fish of the fish j is decreased by 1, and step S7 is performed;
in step S7, it is determined whether all the fish in the frame image i have been determined, if yes, the process proceeds to step S8, otherwise, j is made j +1, and the process returns to step S4;
in step S8, it is determined whether all the frame images extracted during the period t have been determined to be completed, if yes, t is made t +1, and the process returns to step S1, otherwise, i is made i +1, and the process returns to step S3.
In one embodiment of the present invention, the training method of the Faster-RCNN is as follows:
acquiring a fish passing video acquired in a set time period in a fish passing channel, and performing frame extraction processing on the fish passing video to obtain a plurality of frame images A;
marking fish in all frame images A, performing noise reduction processing on the marked frame images A, and then adjusting each frame image A into a picture with the size of 720 × 406 pixels;
and dividing all the pictures with the adjusted sizes into a training set and a testing set according to a set proportion, and then training the Faster-RCNN by adopting the training set and the testing set to obtain the trained Faster-RCNN.
The method for training the fast-RCNN by adopting the training set comprises the following steps:
inputting the pictures in the training set into a convolutional neural network for feature extraction to obtain a feature map;
generating an Anchor box by using RPN, then cutting and filtering the feature image, and judging the foreground and the background of the feature image by using a softmax classifier;
mapping the suggested window after the regression correction of the anchor box of the bounding box to the last layer of feature map of the convolutional neural network, and generating feature maps with preset sizes for each ROI through an ROI pooling layer;
and finally, performing joint training on the classification probability and the frame regression by using the detection classification probability and the detection frame regression respectively.
The scheme also comprises the step of adjusting each frame image to be 720 × 406 pixel before inputting the frame image into the fast-RCNN. The unique setting of the picture size not only can effectively improve the speed of processing data by a computer and save the memory, but also can improve the accuracy of a neural network.
The accuracy of the fish number identification of the scheme is described by combining the test example as follows:
selecting three fishes with larger body type differences, adding a 10-watt LED lamp to fill in an aquarium with the length, the width and the height of 300mm, 150mm and 20mm respectively, recording video information for one minute by using a video camera with 4000 ten thousand pixels, and extracting frames of the video according to 30 frames per second to obtain picture information; the total number of the tests is 6, 10, 20, 12, 17, 20 and 10 fishes are respectively placed in the fish tank every time, after the video images are obtained, the scheme is adopted to count the quantity of the fishes, and the identification result is shown in table 2.
Table 26 test detection accuracy
As can be seen from the table 2, the intelligent fish monitoring method of the scheme has high accuracy in fish quantity statistics.