US11861822B2 - Image recognition device and method for retrieving information on a marker - Google Patents

Abstract

An image recognition device and method retrieve information on a marker. The marker encodes information identifying an asset by encoding the information in a binary pattern as recesses and non-recesses in the marker. The image recognition is performed by imaging the marker, mapping contrast in the image, identifying variations in the contrast, creating a mesh overlaying the image, identifying the present or absence of recesses from the mesh, and reading the binary pattern represented by the recesses in the marker.

Description

FIELD OF THE DISCLOSURE

The present disclosure relates generally to image recognition, and, more particularly, to an image recognition device and method for retrieving information on a marker.

BACKGROUND OF THE DISCLOSURE

Binary information has been encoded in holes on cards for over a century. One notable example is a card with holes used in Jacquard looms, which utilize such recesses to control the threading of fabric. Such holes can encode information to perform the control of the threading. The card in a Jacquard loom can be composed of metal for durability, and so such cards can be radiopaque by virtue of being metallic.

Other cards encoding information using holes are punch cards which can control computers. Using a hole punch device or other similar mechanisms, a programmer can selectively punch the holes into the card. In turn, readers are known for extracting the information encoded by the holes. Some readers utilize mechanical actuators which physically detect the presence or absence of holes in the card.

SUMMARY OF THE DISCLOSURE

According to an embodiment consistent with the present disclosure, an image recognition device and method retrieve information on a marker. The marker encodes information identifying an asset by encoding the information in a binary pattern as recesses and non-recesses in the marker.

In one embodiment, an image recognition device of the present disclosure comprises a camera, a processor, and a memory. The camera is configured to capture an image of a marker. The marker has recesses in at least a top surface of the marker in a field of view of the camera. The processor is configured to receive the image from the camera. The memory is accessible by the processor, with the memory configured to store a plurality of modules each comprising code executable by the processor. The plurality of modules comprises a contrast mapping module, a contrast variation identifying module, a mesh creation module, and a reading module. The contract mapping module is configured to map contrast in the image to generate a set of contrasts from the image. The contrast variation identifying module is configured to identify variations in the set of contrasts. The mesh creation module is responsive to the variations and is configured to create a mesh overlaying the image to locate the position of all possible recesses in the image to be in a transformed image. The reading module is configured to read, from the transformed image, a binary pattern represented by the recesses. The marker can be composed of a radiopaque material, and the camera captures an X-ray radiograph of the marker as the image. The recesses can be holes extending through the top surface and a bottom surface of the marker. The contrast is an optical contrast of optical features in the image. Alternatively, the contrast is a variation of a parameter to encode the binary pattern in the recesses in the marker.

The variations in the contrast are determined relative to a predetermined threshold. Alternatively, the variations in the contrast are determined relative to illumination of the marker during the image capturing by the camera. In a further alternative embodiment, the variations in the contrast are determined relative to exposure of the marker during the image capturing by the camera. The mesh has equally spaced lines. Alternatively, the mesh has irregularly spaced lines.

In another embodiment, a method of the present disclosure comprises capturing an image of a marker using a camera. The marker has recesses in at least a top surface in the field of view of the camera. The method further includes mapping contrast in the image, identifying variations in the contrast, and responsive to the variations in creating a mesh overlaying the image. The method then locates the position of recesses in the image, identifies the presence or absence of recesses in the image, and reads a binary pattern represented by the recesses.

The marker is composed of a radiopaque material, and the camera captures an X-ray radiograph of the marker as the image. The recesses can be holes extending through the top surface and a bottom surface of the marker. The contrast is an optical contrast of optical features in the image. Alternatively, the contrast is a variation of a parameter to encode the binary pattern in the recesses of the marker. The variations in the contrast are determined relative to a predetermined threshold. Alternatively, the variations in the contrast are determined relative to illumination of marker during the image capturing by the camera. In a further alternative embodiment, the variations in the contrast are determined relative to exposure of marker during the image capturing by the camera. The mesh has equally spaced lines. Alternatively, the mesh has irregularly spaced lines.

Any combinations of the various embodiments and implementations disclosed herein can be used in a further embodiment, consistent with the disclosure. These and other aspects and features can be appreciated from the following description of certain embodiments presented herein in accordance with the disclosure and the accompanying drawings and claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG.1 illustrates an image recognition device scanning a binary punch marker, according to an embodiment.

FIG.2 is a schematic diagram of the image recognition device ofFIG.1.

FIG.3 is a top front side view of the binary punch marker.

FIG.4 illustrates deformation of the marker and its recesses.

FIG.5 illustrates occlusion of recesses.

FIG.6 illustrates overlaying of a mesh to determine the presence or absence of recesses.

FIG.7 is a flowchart of an image recognition method, according to the embodiment.

It is noted that the drawings are illustrative and are not necessarily to scale.

DETAILED DESCRIPTION OF CERTAIN EMBODIMENTS OF THE DISCLOSURE

Example embodiments consistent with the teachings included in the present disclosure are directed to an image recognition device and method which retrieve information on a marker by imaging recesses on the marker.

According to an embodiment shown inFIG.1, animage recognition device10 captures and processes an image of amarker12 havingrecesses13 and non-recesses15, as shown inFIGS.1 and3. Themarker12 can be fastened to an asset such as apipe60. Other examples of assets include pipelines, storage tanks, etc. Themarker12 can be fastened adjacent to a weld joint orseam62 of thepipe60. As used herein, the term “joint” refers to a joining of two pipe members to create an overall pipe. Theimage recognition device10 captures the image of themarker12 withrecesses13 within a field of view (FOV)32 of acamera28 of thedevice10. By capturing and processing the image, thedevice10 extracts information identifying theweld seam62.

Therecesses13 andnon-recesses15 encode the information identifying theweld seam62. Alternatively, the information identifies the asset itself. The information can be a binary pattern which encodes the information. In certain embodiments, the binary pattern can be further encrypted. In a particular implementation consistent with this aspect of the disclosure, the further encryption is performed using an obfuscation procedure. The obfuscation procedure can be a hash function, for example, using a hash table. As will be understood by those of skill in the art, when further obfuscation is to be used, a different encryption process can be employed, including any known operation on the information to obfuscate it.

Referring toFIGS.1-2, theimage recognition device10 includes aprocessor20, amemory22, aninput device24, anoutput device26, and thecamera28. Thememory22 is accessible by theprocessor20. Thememory22 can storepredetermined software30 including a plurality of modules each comprising code which is executed by theprocessor20 to operate theimage recognition device10. Thepredetermined software30 can be composed of at least one module configured to perform a specific function, as described below. Theinput device24 can include a keyboard, a keypad, a mouse, a touchscreen, or combinations thereof. Theoutput device26 can include a display. The display can be incorporated into a touchscreen. Theinput device24 and theoutput device26 can implement a graphic user interface. Theimage recognition device10 can be implemented in any computing device, such as a handheld device. For example, theimage recognition device10 can be implemented in a smartphone, and thepredetermined software30 can be an app. Alternatively, theimage recognition device10 can be a tablet or a desktop personal computer (PC) running an application implementing thepredetermined software30.

Referring toFIG.3, therecesses13 can extend through at least atop surface52 of the substrate of themarker12. Alternatively, therecesses13 are holes which extend through both thetop surface52 and abottom surface54. The substrate can be composed of a radiopaque material, including metal such as lead. Accordingly, thecamera28 can include a radiographic imager, such as an X-ray camera. By using a radiopaque material, themarker12 can be used during inspection of pipes and welds at joints employing radiography such as X-rays, since themarker12 will scatter X-rays. Alternatively, thecamera28 can be an optical camera operating using visible light to capture an image of therecesses13 andnon-recesses15 in visible light wavelengths.

Thebinary punch marker12 can be generally cuboidal in shape with a rectangular cross-section as shown inFIG.3. It is understood that themarker12 can have any other shape. It is also understood that themarker12 can have any other cross-section such as a circular cross-section. Therecesses13 can be circular or any shape, such as a polygon. Themarker12 has athickness56, as shown inFIG.3. Thethickness56 can be adjusted to the thickness of the asset, such as the underlying pipe to which themarker12 is attached. The adjustment of thethickness56 can provide sufficient contrast of therecesses13 relative to the asset. For example, thethickness56 can be less than 1 mm and having sufficient contrast. Such athickness56 hasrecesses13 with sufficient contrast even with themarker12 placed on top of several centimeters of steel.

When themarker12 is placed on the curved surface, such as the surface of thepipe60, themarker12 can deform to complement a general curved shape of the surface, since themarker12 is composed of metal such as lead. Accordingly, as shown inFIG.4, although afirst set70 ofrecesses13 can remain substantially circular, asecond set72 ofrecesses13 can deform to not be fully circular. In addition, thecamera28 views theset70 ofrecesses13 substantially straight on with no angular displacement. However, due to the complementing of themarker12 to the curved shape of the surface, thecamera28 views theset72 ofrecesses13 at an angle, and so the set72 ofrecesses13 do not appear fully circular. Accordingly, as shown inFIGS.4-5, therecess13 appears to thecamera28 as anarrow recess74 withpartial occlusion76.

When processing the image of therecesses13 andnon-recesses15 in amarker12, theimage recognition device10 overlayshorizontal lines78 andvertical lines80 to form amesh82, as shown inFIG.6. Using themesh82, the locations of therecesses13 and the non-recesses15 can be determined.

As shown inFIG.7, amethod200 is performed for image recognition of information stored byrecesses13 andnon-recesses15 of amarker12. Themethod200 includes thestep210 of scanning themarker12 having therecesses13 andnon-recesses15 using theimage recognition device10. Thedevice10 then generates an image of themarker12. For example, theimage recognition device10 includes thecamera28. Thecamera28 can be a radiographic camera configured to capture an image, for example, using X-rays. Alternatively, thecamera28 can be an optical camera configured to capture an image, for example, using light in visible wavelengths.

Themethod200 also includes thestep220 of mapping contrast in the image using a contrast mapping module to generate a set of contrasts from the image. The contrast can be an optical contrast of optical features in the image. Alternatively, the contrast is a variation of a parameter used to encode the information identifying theweld seam62 as therecesses13 andnon-recesses15 in themarker12.

Themethod200 further includes thestep230 of identifying variations in the contrast using a contrast variation identifying module to identify variations in the set of contrasts. The variations in the contrast can be determined relative to a predetermined threshold. Alternatively, the variations in the contrast can be determined relative to illumination of the marker during the image capturing by thecamera28. In a further embodiment, the variations in the contrast can be determined relative to exposure of marker during the image capturing by thecamera28.

In addition, themethod200 includes thestep240 of creating amesh82 using a mesh creation module. The mesh creation module responds to the variations in the contrast to determine at least onerecess13. Using the at least onerecess13, the mesh creation module is configured to create themesh82. Themesh82 overlays the image to locate the position of allpossible recesses13 in themarker12. Themesh82 can have equally spaced lines. Alternatively, due to deformation of themarker12 as described above, themesh82 can have lines spaced at irregular intervals. For example, referring toFIG.6, the mesh line through theset72 of recesses may be closer to a nearby mesh line than such mesh lines through and near theset70 of recesses.

Themethod200 also includes thestep250 of identifying the presence or absence ofrecesses13 using a recess identifying module. The recess identifying module determines where in the mesh that recesses13 are expected to be. For example, the recess identifying module can measure the contrast along each of thelines78,80 inFIG.6. In one embodiment, if the contrast is greater than a predetermined threshold, the recess identifying module determines that arecess13 is present, as inFIG.3. Otherwise, the recess identifying module determines that a non-recess15 is present, as inFIG.3. The recess identifying module then creates a transformed image withdetermined recesses13 andnon-recesses15.

Themethod200 further includes thestep260 of reading, from the transformed image, the binary pattern of the identifying information represented by therecesses13 andnon-recesses15 in themarker12 using a reading module. The reading module decodes the binary pattern of therecesses13 andnon-recesses15 to extract the information identifying theweld seam62. The reading module can also decrypt the binary pattern if such identifying information has been encrypted. For example, the decryption can be a reverse hash function matching the original hash function. In one embodiment, the reverse hash function can use the original hash table to reverse the original hash function. Alternatively, the decryption can match the encryption process used to encrypt the identifying information.

Portions of the methods described herein can be performed by software or firmware in machine readable form on a tangible (e.g., non-transitory) storage medium. For example, the software or firmware can be in the form of a computer program including computer program code adapted to cause the fabrication system to perform various actions described herein when the program is run on a computer or suitable hardware device, and where the computer program can be embodied on a computer readable medium. Examples of tangible storage media include computer storage devices having computer-readable media such as disks, thumb drives, flash memory, and the like, and do not include propagated signals. Propagated signals can be present in a tangible storage media. The software can be suitable for execution on a parallel processor or a serial processor such that various actions described herein can be carried out in any suitable order, or simultaneously.

It is to be further understood that like or similar numerals in the drawings represent like or similar elements through the several figures, and that not all components or steps described and illustrated with reference to the figures are required for all embodiments or arrangements.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “contains”, “containing”, “includes”, “including,” “comprises”, and/or “comprising,” and variations thereof, when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

Terms of orientation are used herein merely for purposes of convention and referencing and are not to be construed as limiting. However, it is recognized these terms could be used with reference to an operator or user. Accordingly, no limitations are implied or to be inferred. In addition, the use of ordinal numbers (e.g., first, second, third) is for distinction and not counting. For example, the use of “third” does not imply there is a corresponding “first” or “second.” Also, the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of “including,” “comprising,” “having,” “containing,” “involving,” and variations thereof herein, is meant to encompass the items listed thereafter and equivalents thereof as well as additional items.

While the disclosure has described several exemplary embodiments, it will be understood by those skilled in the art that various changes can be made, and equivalents can be substituted for elements thereof, without departing from the spirit and scope of the invention. In addition, many modifications will be appreciated by those skilled in the art to adapt a particular instrument, situation, or material to embodiments of the disclosure without departing from the essential scope thereof. Therefore, it is intended that the invention not be limited to the particular embodiments disclosed, or to the best mode contemplated for carrying out this invention, but that the invention will include all embodiments falling within the scope of the appended claims.

The subject matter described above is provided by way of illustration only and should not be construed as limiting. Various modifications and changes can be made to the subject matter described herein without following the example embodiments and applications illustrated and described, and without departing from the true spirit and scope of the invention encompassed by the present disclosure, which is defined by the set of recitations in the following claims and by structures and functions or steps which are equivalent to these recitations.

Claims (20)

What is claimed is:

1. An image recognition device comprising:

a camera configured to capture an image of a marker having recesses in at least a top surface of the marker in a field of view of the camera, wherein the camera is selected from the group consisting of: an optical camera and an X-ray camera;

a processor configured to receive the image from the camera; and

a memory accessible by the processor, with the memory configured to store a plurality of modules each comprising code executable by the processor, the plurality of modules comprising:

a contrast mapping module configured to map contrast in the image to generate a set of contrasts from the image;

a contrast variation identifying module configured to identify variations in the set of contrasts;

a mesh creation module responsive to the variations and configured to create a mesh overlaying the image to locate the position of all possible recesses in the image to be in a transformed image; and

a reading module configured to read, from the transformed image, a binary pattern represented by the recesses.

2. The image recognition device ofclaim 1, wherein the marker is composed of a radiopaque material, and wherein the camera captures an X-ray radiograph of the marker as the image.

3. The image recognition device ofclaim 1, wherein the recesses are holes extending through the top surface and a bottom surface of the marker.

4. The image recognition device ofclaim 1, wherein the contrast is an optical contrast of optical features in the image.

5. The image recognition device ofclaim 1, wherein the contrast is a variation of a parameter to encode the binary pattern in the recesses in the marker.

6. The image recognition device ofclaim 1, wherein the variations in the contrast are determined relative to a predetermined threshold.

7. The image recognition device ofclaim 1, wherein the variations in the contrast are determined relative to illumination of the marker during the image capturing by the camera.

8. The image recognition device ofclaim 1, wherein the variations in the contrast are determined relative to exposure of the marker during the image capturing by the camera.

9. The image recognition device ofclaim 1, wherein the mesh has equally spaced lines.

10. The image recognition device ofclaim 1, wherein the mesh has irregularly spaced lines.

11. A method comprising:

capturing an image of a marker using a camera, the marker having recesses in at least a top surface in the field of view of the camera, wherein the camera is selected from the group consisting of: an optical camera and an X-ray camera;

mapping contrast in the image;

identifying variations in the contrast;

responsive to the variations, creating a mesh overlaying the image;

locating the position of recesses in the image;

identifying the presence or absence of recesses in the image; and

reading a binary pattern represented by the recesses.

12. The method ofclaim 11, wherein the marker is composed of a radiopaque material, and

wherein the camera captures an X-ray radiograph of the marker as the image.

13. The method ofclaim 11, wherein the recesses are holes extending through the top surface and a bottom surface of the marker.

14. The method ofclaim 11, wherein the contrast is an optical contrast of optical features in the image.

15. The method ofclaim 11, wherein the contrast is a variation of a parameter to encode the binary pattern in the recesses of the marker.

16. The method ofclaim 11, wherein the variations in the contrast are determined relative to a predetermined threshold.

17. The method ofclaim 11, wherein the variations in the contrast are determined relative to illumination of marker during the image capturing by the camera.

18. The method ofclaim 11, wherein the variations in the contrast are determined relative to exposure of marker during the image capturing by the camera.

19. The method ofclaim 11, wherein the mesh has equally spaced lines.

20. The method ofclaim 11, wherein the mesh has irregularly spaced lines.

Images