技术领域Technical Field
本公开涉及2D图像捕获、图像处理、以及3D或多维图像或图像序列的模拟显示。The present disclosure relates to 2D image capture, image processing, and simulated display of 3D or multi-dimensional images or image sequences.
背景技术Background technique
人类视觉系统(HVS)依靠二维图像来解释三维视场。通过利用具有HVS的机制，我们创建了与HVS兼容的图像/场景。The Human Visual System (HVS) relies on 2D images to interpret 3D fields of view. By leveraging mechanisms with the HVS, we create images/scenes that are compatible with the HVS.
在观看3D图像时，在眼睛必须会聚的点与眼睛必须聚焦到的距离之间的不匹配会产生负面的后果。虽然3D影像已被证明在电影、数字广告方面很受欢迎且有用，但如果观看者能够在不佩戴专门的眼镜或头戴式耳机的情况下观看3D图像，许多其他的应用也可能被利用，这是众所周知的问题。在观看数字多维图像时，这些系统中的不对准会导致图像跳动、失焦或模糊的特征。观看这些图像会导致头疼和恶心。When viewing 3D images, the mismatch between the point at which the eye must converge and the distance to which the eye must focus can have negative consequences. While 3D imaging has proven popular and useful in movies, digital advertising, and many other applications could be exploited if viewers were able to view 3D images without wearing specialized glasses or headphones, which is a well-known problem. Misalignments in these systems can cause image jumps, out-of-focus or blurry features when viewing digital multi-dimensional images. Viewing these images can cause headaches and nausea.
在自然观看中，图像到达眼睛时有不同的双眼视差，因此当观看者从视觉场景中的一个点看向另一个点时，他们必须调整眼睛的辐辏(vergence，聚散)。视线相交处的距离就是辐辏距离。不能在这个距离上会聚会出现双重图像。观看者还为场景的固定部分适当地调整每只眼睛晶状体的焦度(即调节(accommodate))。眼睛必须聚焦到的距离就是调节距离。不能调节到这个距离会出现模糊的图像。辐辏和调节反应在大脑中是耦合的，具体地，辐辏的变化驱动调节的变化，调节的变化驱动辐辏的变化。这种耦合在自然观看中是有利的，因为辐辏距离和调节距离几乎总是相同的。In natural viewing, images arrive at the eyes with different binocular disparities, so as viewers look from one point in the visual scene to another, they must adjust the vergence of the eyes. The distance at which the lines of sight meet is the vergence distance. Failure to converge at this distance results in a double image. Viewers also adjust the power of the lens of each eye appropriately for a fixed portion of the scene (i.e., accommodation). The distance to which the eyes must focus is the accommodation distance. Failure to accommodate to this distance results in a blurred image. The vergence and accommodation responses are coupled in the brain, specifically, changes in vergence drive changes in accommodation, and changes in accommodation drive changes in vergence. This coupling is advantageous in natural viewing because the vergence distance and the accommodation distance are almost always the same.
在3D图像中，图像具有不同的双眼视差，从而刺激了如自然观看中发生的辐辏变化。但调节距离仍然固定在与观看者的显示距离处，因此，辐辏距离和调节距离之间的自然相关性被破坏了，导致所谓的辐辏-调节冲突。该冲突导致了几个问题。首先，不同的视差和聚焦信息会导致感知上的深度失真。第二，观看者在同时定影(fuse)和聚焦于图像中的关键主体时遇到困难。最后，试图分别调整辐辏和调节会导致观看者的视觉不适和疲劳。In 3D images, the images have different binocular disparities, stimulating a change in vergence as occurs in natural viewing. But the accommodation distance remains fixed at the display distance from the viewer, so the natural correlation between the vergence distance and the accommodation distance is destroyed, resulting in the so-called vergence-accommodation conflict. This conflict causes several problems. First, the different disparity and focus information can cause perceived depth distortion. Second, the viewer has difficulty simultaneously fusing and focusing on key subjects in the image. Finally, trying to adjust vergence and accommodation separately can cause visual discomfort and fatigue to the viewer.
对深度的感知是基于各种线索的，双眼视差(binocular disparity)和运动视差(motion parallax)通常比图像线索提供更精确的深度信息。双眼视差和运动视差为深度感知提供了两个独立的定量线索。双眼视差指的是一个点在3D空间中的两个视网膜图像投影之间的位置差异。The perception of depth is based on various cues, and binocular disparity and motion parallax usually provide more accurate depth information than image cues. Binocular disparity and motion parallax provide two independent quantitative cues for depth perception. Binocular disparity refers to the difference in position between two retinal image projections of a point in 3D space.
传统的立体显示器迫使观看者试图将这些过程解耦，因为当他们必须动态地改变辐辏角以观看不同立体距离处的对象时，他们必须将调节保持在一个固定的距离，否则整个显示器将滑出焦点。在观看这种显示器时，这种解耦会产生眼睛疲劳，并损害图像质量。Conventional stereoscopic displays force viewers to try to decouple these processes because while they must dynamically change the angle of convergence to view objects at different stereoscopic distances, they must maintain accommodation at a fixed distance or the entire display will slide out of focus. This decoupling can produce eye fatigue and impair image quality when viewing such displays.
最近，一部分摄影师正在利用20世纪80年代的相机，诸如NIMSLO和NASHIKA 3D35mm模拟胶片相机或在多个点之间移动的数码相机来拍摄场景的多个帧，从模拟相机中冲洗多个帧的胶片，将图像上传到图像软件，诸如PHOTOSHOP，并布置图像以创建摆动图、移动的GIF效果。Recently, some photographers are using cameras from the 1980s, such as the NIMSLO and NASHIKA 3D 35mm analog film cameras or digital cameras that move between multiple points to shoot multiple frames of a scene, develop the multiple frames of film from the analog camera, upload the images to image software such as PHOTOSHOP, and arrange the images to create a wiggling, moving GIF effect.
因此，明显的是，对于具有集成的2D数字图像捕获系统、图像操纵应用和3D数字图像或图像序列的显示的智能装置，存在着明显的未满足的需求，该装置可以被配置用于解决以上讨论的问题的至少一些方面。It is therefore apparent that there is a significant unmet need for a smart device with an integrated 2D digital image capture system, image manipulation application, and display of 3D digital images or image sequences that can be configured to address at least some aspects of the issues discussed above.
发明内容Summary of the invention
简而言之，在示例实施方式中，本公开可以克服上面所提及的缺点，并且本公开可以满足对2D图像捕获系统以及3D数字图像和3D序列的显示的公认需求，智能装置具有：存储器装置，该存储器装置用于存储指令；处理器，该处理器与存储器进行通信，并且该处理器被配置成执行该指令；多个数字图像捕获装置，该多个数字图像捕获装置与处理器进行通信，并且每个图像捕获装置被配置成捕获场景的数字图像，该多个数字图像捕获装置以线性串联的方式定位在大约瞳孔间距离宽度内，其中，第一数字图像捕获装置在接近瞳孔间距离宽度的第一端部处居中，第二数字图像捕获装置在接近瞳孔间距离宽度的第二端部处居中，并且任何剩余的多个数字图像捕获装置均匀地间隔在第一数字图像捕获装置与第二数字图像捕获装置之间，处理步骤，以配置数据集；以及显示器，该显示器被配置成显示所模拟的多维数字图像序列和/或多维数字图像，经由区块链存储多维数字图像序列和/或多维数字图像，存储音频文件以便在查看图像文件时播放，存储认证图像或音频文件的相关验证文档，传输存储的文件，对此类文件进行区块链存储，创建此类存储文件的不可替代资产、不可替代令牌(NFT)。In short, in example embodiments, the present disclosure may overcome the above-mentioned disadvantages, and the present disclosure may meet the recognized needs for 2D image capture systems and displays of 3D digital images and 3D sequences, a smart device having: a memory device for storing instructions; a processor in communication with the memory and configured to execute the instructions; a plurality of digital image capture devices in communication with the processor, and each image capture device is configured to capture a digital image of a scene, the plurality of digital image capture devices being positioned in a linear series within approximately the width of the interpupillary distance, wherein a first digital image capture device is positioned within approximately the width of the interpupillary distance, wherein a first digital image capture device is positioned within approximately the width of the interpupillary distance, wherein a second ... The first digital image capture device is centered at the end, the second digital image capture device is centered at the second end proximate the width of the interpupillary distance, and any remaining multiple digital image capture devices are evenly spaced between the first digital image capture device and the second digital image capture device, a processing step to configure a data set; and a display configured to display the simulated multi-dimensional digital image sequence and/or multi-dimensional digital image, store the multi-dimensional digital image sequence and/or multi-dimensional digital image via blockchain, store audio files for playback when viewing image files, store related verification documents for authenticating images or audio files, transmit stored files, store such files on blockchain, and create non-fungible assets, non-fungible tokens (NFTs) for such stored files.
因此，该系统和使用方法的特征是：它具有能够利用2D捕获装置捕获场景的多个图像的能力，所述2D捕获装置定位在大约相隔眼内或瞳孔间距离宽度IPD(人类视觉系统的瞳孔之间的距离)。Thus, the system and method of use are characterized by the ability to capture multiple images of a scene using 2D capture devices positioned approximately an intraocular or interpupillary distance width IPD (the distance between the pupils of the human visual system).
因此，系统和使用方法的特征是它能够将输入的2D源图像转换成多维/多光谱图像序列。输出的图像遵循“关键主体点”保持在最佳视差内的规则，以保持清晰明了的图像。Therefore, the system and method of use are characterized in that it is capable of converting an input 2D source image into a multi-dimensional/multi-spectral image sequence. The output image follows the rule that the "key subject points" are kept within the optimal parallax to maintain a clear and crisp image.
因此，该系统和使用方法的特征是：它具有能够利用现有的观看装置来显示模拟的多维数字图像序列的能力。Thus, the system and method of use are characterized by the ability to display simulated multi-dimensional digital image sequences using existing viewing equipment.
因此，该系统和使用方法的特征是：它具有能够拍摄多维数字图像序列、观看多维数字图像序列和通过互联网发送多维数字图像序列的能力。这个独立的系统可以集成到智能手机、平板电脑中，或与外部装置一起使用。一系列的4个相机镜头使我们能够产生特殊的运动视差图像——DIGY，该图像不需要特殊的屏幕就可以观看。该系统可以在全自动模式下或者在手动模式下使用，以供操作者与该场景的互动。Thus, the system and method of use are characterized by the ability to capture, view and send multidimensional digital image sequences via the Internet. This stand-alone system can be integrated into a smartphone, tablet, or used with an external device. A series of 4 camera lenses allows us to produce special motion parallax images - DIGY, which do not require a special screen to view. The system can be used in fully automatic mode or in manual mode for the operator to interact with the scene.
因此，系统和使用方法的特征是以下能力：将观看装置或其他观看功能集成到显示器中，诸如在LED或OLED、LCD、OLED及其组合或其他观看装置中具有集成LCD层的屏障屏幕(黑线)、双凸透镜、弧形、曲面、梯形、抛物线、覆盖、波导、黑线等。Thus, the systems and methods of use are characterized by the ability to integrate viewing devices or other viewing functionality into displays, such as barrier screens (black lines), lenticular lenses, arcs, curved surfaces, trapezoids, parabolas, overlays, waveguides, black lines, etc. with an integrated LCD layer in LED or OLED, LCD, OLED and combinations thereof, or other viewing devices.
基于数字多维图像平台的系统和使用方法的另一个特征是：产生数字多维图像的能力，这些生数字多维图像可以在观看屏幕上观看，诸如移动电话和固定电话、智能手机(包括iPhone)、平板电脑、电脑、笔记本电脑、监视器和其他显示器和/或特定输出装置，直接不需要3D眼镜或耳戴式耳机。Another feature of the systems and methods of use based on the digital multi-dimensional image platform is the ability to generate digital multi-dimensional images that can be viewed on viewing screens such as mobile phones and landline phones, smartphones (including iPhones), tablets, computers, laptops, monitors and other displays and/or specific output devices without the need for 3D glasses or ear-worn headphones.
在示例性实施方式中，一种根据场景的一系列2D图像来模拟3D图像序列的系统，该系统包括智能装置，该智能装置具有：存储器装置，该存储器装置用于存储指令；处理器，该处理器与所述存储器装置进行通信，该处理器被配置成执行所述指令；多个数字图像捕获装置，该多个数字图像捕获装置与所述处理器进行通信，并且该多个图像捕获装置中的每个图像捕获装置被配置成捕获场景的数字图像，所述多个数字图像捕获装置以线性串联的方式大约定位在大约瞳孔间距离宽度内，其中，第一数字图像捕获装置定位在瞳孔间距离宽度的第一端部附近，第二数字图像捕获装置定位在瞳孔间距离宽度的第二端部附近，并且任何剩余的所述多个数字图像捕获装置均匀地间隔在第一数字图像捕获装置与第二数字图像捕获装置之间以捕获场景的一系列2D图像；显示器，该显示器与所述处理器进行通信，所述显示器被配置成显示所述多维数字图像序列，并且经由所述显示器上的输入将音频文件叠加在所述多维数字图像序列上。In an exemplary embodiment, a system for simulating a sequence of 3D images from a series of 2D images of a scene includes an intelligent device having: a memory device for storing instructions; a processor in communication with the memory device, the processor configured to execute the instructions; a plurality of digital image capture devices in communication with the processor, each of the plurality of image capture devices configured to capture a digital image of a scene, the plurality of digital image capture devices being positioned approximately within an interpupillary distance width in a linear series, wherein a first digital image capture device is positioned near a first end of the interpupillary distance width, a second digital image capture device is positioned near a second end of the interpupillary distance width, and any remaining digital image capture devices of the plurality of digital image capture devices are evenly spaced between the first digital image capture device and the second digital image capture device to capture a series of 2D images of the scene; a display in communication with the processor, the display configured to display the multi-dimensional digital image sequence, and to overlay an audio file on the multi-dimensional digital image sequence via an input on the display.
本公开的特征可以包括一种系统，该系统具有一系列捕获装置，诸如两个、三个、四个或更多个，这样的多个捕获装置(数字图像相机)在眼内或瞳孔间距离宽度——普通人的瞳孔之间的距离——内线性定位，该系统捕获且存储两个、三个、四个或更多个，多个场景的2D源图像，该系统基于捕获该图像的源捕获装置来标记和识别这些图像。Features of the present disclosure may include a system having a series of capture devices, such as two, three, four or more, such multiple capture devices (digital image cameras) are linearly positioned within the eye or within the width of the interpupillary distance - the distance between the pupils of an average person - the system captures and stores two, three, four or more, 2D source images of multiple scenes, the system tags and identifies these images based on the source capture device that captured the image.
本公开的特征可以包括一种系统，该系统具有由诸如顶部玻璃盖、电容式触摸屏玻璃、偏振器、漫射器和背光的部件的堆配置的显示装置。此外，图像源，诸如LCD，诸如LED、ELED、PDP、QLED，以及其他类型的显示技术。此外，显示装置可以包括透镜阵列，该透镜阵列优选位于电容式触摸屏玻璃与LCD面板部件堆之间，并配置成以能够显示高质量2D图像和作为场景的3D或多维数字图像的左右交错立体图像对的方式弯曲或折射光。Features of the present disclosure may include a system having a display device configured from a stack of components such as a top glass cover, a capacitive touch screen glass, a polarizer, a diffuser, and a backlight. In addition, an image source such as an LCD, such as an LED, an ELED, a PDP, a QLED, and other types of display technologies. In addition, the display device may include a lens array, which is preferably located between the capacitive touch screen glass and the LCD panel component stack and is configured to bend or refract light in a manner that enables the display of high-quality 2D images and left-right interlaced stereoscopic image pairs as a 3D or multi-dimensional digital image of a scene.
本公开的特征是：通过用于确定会聚点或关键主体点的另一个重要的参数来克服上述缺陷的能力，因为观看没有与关键主体点对准的图像会对人类视觉系统造成混乱，导致图像模糊和双重图像。A feature of the present disclosure is the ability to overcome the above mentioned drawbacks through another important parameter for determining the convergence point or critical subject point, since viewing an image that is not aligned with the critical subject point can be confusing to the human visual system, resulting in blurred images and double images.
本公开的特征是：能够在近平面或接近平面和远平面或后平面之间的任何地方选择会聚点或关键主体点的能力，手动模式用户选择。Features of the present disclosure are: the ability to select a convergence point or key subject point anywhere between the near or approach plane and the far or back plane, with manual mode user selection.
本公开的特征是：通过用于确定舒适圆CoC的另一个重要的参数来克服上述缺陷的能力，因为观看没有与舒适圆CoC对准的图像会对人类视觉系统造成混乱，导致图像模糊和双重图像。A feature of the present disclosure is the ability to overcome the above-mentioned drawbacks through another important parameter for determining the comfort circle CoC, since viewing images that are not aligned with the comfort circle CoC can cause confusion to the human visual system, resulting in image blur and double images.
本公开的特征是：通过用于确定与同视点界弧或点和帕努姆区域相融合的舒适圆CoC的另一个重要参数来克服上述缺陷的能力，因为观看没有对准与和同视点弧或点和Panum区域相融合的舒适圆CoC的图像会对人的视觉系统造成混乱，导致图像模糊和双重图像。A feature of the present disclosure is the ability to overcome the above-mentioned defects by using another important parameter for determining the comfort circle CoC that merges with the isotropy arc or point and the Panum's area, because viewing an image that is not aligned with the comfort circle CoC that merges with the isotropy arc or point and the Panum's area will cause confusion to the human visual system, resulting in blurred images and double images.
本公开的特征是：通过用于确定灰度深度图的另一个重要参数来克服上述缺陷的能力，该系统基于场景中的指定点(最近点、关键主体点和最远点)来插值中间点，该系统为这些中间点分配值，并且将其总和渲染成灰度深度图。该灰度图使用分配给场景中不同点(最近点、关键主体点和最远点)的值来生成体积视差。这种模式还允许将体积视差或圆角分配给场景中的单一对象。The feature of the present disclosure is the ability to overcome the above-mentioned deficiencies through another important parameter for determining the grayscale depth map. The system interpolates intermediate points based on specified points in the scene (closest point, key subject point, and farthest point). The system assigns values to these intermediate points and renders their sum into a grayscale depth map. The grayscale map uses the values assigned to different points in the scene (closest point, key subject point, and farthest point) to generate volumetric disparity. This mode also allows volumetric disparity or rounded corners to be assigned to a single object in the scene.
本公开的特征是其测量对象或对象元素的深度或z轴和/或基于场景中对象的已知大小进行比较的能力。A feature of the present disclosure is its ability to measure the depth or z-axis of an object or element of an object and/or make comparisons based on the known size of objects in the scene.
本公开的特征是：它具有能够利用关键主体算法来在显示器上显示的场景的多个图像中手动或自动选择关键主体并产生多维数字图像序列以供在显示器上观看的能力。A feature of the present disclosure is its ability to utilize a key subject algorithm to manually or automatically select a key subject among multiple images of a scene displayed on a display and generate a multi-dimensional digital image sequence for viewing on the display.
本公开的特征是它具有下述能力：利用图像对准、水平图像平移或编辑算法来手动地或自动地将场景的多个图像围绕关键主体对准，以供显示。A feature of the present disclosure is its ability to manually or automatically align multiple images of a scene around a key subject for display using image alignment, horizontal image translation, or editing algorithms.
本公开的特征的特征是：它利用图像平移算法来对准场景的两个图像的关键主体点以供显示的能力。A feature of the present disclosure is its ability to utilize an image translation algorithm to align key subject points of two images of a scene for display.
本公开的特征是：它具有能够生成DIFYS(差分图像格式)的能力，DIFYS是一种用于获得场景的多视图并创建在没有眼镜或任何其他观看辅助工具的情况下产生深度的一系列图像的特定的技术。该系统利用水平图像平移连同一种运动视差的形式来创建3D视图。DIFYS是通过观察者的眼睛对单一场景的不同视图进行翻转而产生的。这些视图是通过图像捕获系统的运动或通过多个相机拍摄的场景来捕获的，其中阵列中的每个相机在不同的位置观看。A feature of the present disclosure is that it has the ability to generate DIFYS (Differential Image Format), which is a specific technique for obtaining multiple views of a scene and creating a series of images that produce depth without glasses or any other viewing aids. The system uses horizontal image translation along with a form of motion parallax to create a 3D view. DIFYS is produced by flipping different views of a single scene through the observer's eyes. These views are captured by the motion of the image capture system or by the scene being photographed by multiple cameras, where each camera in the array views at a different position.
根据本公开的第一方面，根据从2D图像帧序列模拟3D图像序列可以用于从多个不同观察点捕获场景的多个2D图像帧(图像)，其中在序列中的每个图像帧内识别第一近侧平面和第二远侧平面，并且其中每个观察点基本上维持每个图像帧的相同第一近侧图像平面；确定序列中每个图像帧内的第一近侧平面和第二远侧平面的深度估计，对准序列中每个图像帧的第一近侧平面，并基于每个图像帧的第二远侧平面的深度估计移位序列中每个后续图像帧的第二远侧平面，以产生对应于每个2D图像帧的修改后的图像帧并按顺序显示修改后的图像帧。According to a first aspect of the present disclosure, a simulation of a 3D image sequence from a 2D image frame sequence can be used to capture multiple 2D image frames (images) of a scene from multiple different observation points, wherein a first proximal plane and a second distal plane are identified within each image frame in the sequence, and wherein each observation point substantially maintains the same first proximal image plane for each image frame; depth estimates of the first proximal plane and the second distal plane within each image frame in the sequence are determined, the first proximal plane of each image frame in the sequence is aligned, and the second distal plane of each subsequent image frame in the sequence is shifted based on the depth estimate of the second distal plane of each image frame to generate a modified image frame corresponding to each 2D image frame and display the modified image frames in sequence.
本公开改变了显示场景中不同平面处的物体的焦点，以匹配辐辏和立体视网膜视差的需求，以更好地模拟自然观看条件。通过调整场景中关键物体的焦点以匹配它们的立体视网膜视差，使用于眼球调节和辐辏的线索得到一致。如同在自然视觉中，观看者通过改变调节性使不同的物体成为焦点。由于调节和辐辏之间的不匹配减少了，自然观看条件得到了更好的模拟，并且眼睛的疲劳也减少了。The present disclosure changes the focus of objects at different planes in the display scene to match the requirements of convergence and stereoscopic retinal disparity to better simulate natural viewing conditions. By adjusting the focus of key objects in the scene to match their stereoscopic retinal disparity, the cues for eye accommodation and convergence are consistent. As in natural vision, the viewer brings different objects into focus by changing accommodation. Because the mismatch between accommodation and convergence is reduced, natural viewing conditions are better simulated and eye fatigue is reduced.
本公开可以用于确定序列中每个图像帧的三个或更多个平面。The present disclosure may be used to determine three or more planes for each image frame in a sequence.
此外，优选的是，这些平面具有不同的深度估计。Furthermore, it is preferred that the planes have different depth estimates.
此外，优选的是，每个各自的平面基于各自平面的深度估计和第一近侧平面之间的差异进行移位。Furthermore, it is preferred that each respective plane is displaced based on a difference between the depth estimate of the respective plane and the first proximal plane.
优选的是，对每个经修改的图像帧的第一近侧平面进行对准，使得第一近侧平面被定位在同一像素空间。Preferably, the first near side plane of each modified image frame is aligned such that the first near side plane is positioned at the same pixel space.
另外优选的是，第一平面包括关键主体点。It is further preferred that the first plane includes the key subject point.
优选地，这些平面包括至少一个前景平面。Preferably, the planes include at least one foreground plane.
此外，优选的是，这些平面包括至少一个背景平面。Furthermore, it is preferred that the planes include at least one background plane.
优选地，连续的观察点位于一条直线上。Preferably, consecutive observation points are located on a straight line.
根据本发明的第二方面，提供了一种存储指令的非暂态计算机可读存储介质，该指令在由处理器执行时使处理器执行根据本发明的第二方面的方法。According to a second aspect of the present invention, there is provided a non-transitory computer-readable storage medium storing instructions, which when executed by a processor causes the processor to perform a method according to the second aspect of the present invention.
具有2D数字图像捕获系统、图像操纵应用和3D数字图像或图像序列的显示的智能装置的这些和其他特征，对于本领域的技术人员来说，根据鉴于附图或图解阅读的前面的摘要和下面的附图说明、具体实施方式以及权利要求书将变得更加明显。These and other features of a smart device with a 2D digital image capture system, image manipulation application, and display of a 3D digital image or image sequence will become more apparent to those skilled in the art from the foregoing abstract and the following description of the drawings, detailed description, and claims read in light of the accompanying drawings or illustrations.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
通过参照附图阅读优选的和选定的替选实施方式的详细描述，将更好地理解本公开，在附图中，类似的附图标记表示类似的结构，并且自始至终指代类似的元素，并且其中：The present disclosure will be better understood by reading the detailed description of the preferred and selected alternative embodiments with reference to the accompanying drawings, wherein like reference numerals denote like structures and refer to like elements throughout, and wherein:
图1A示出了基于观察者相对于显示器的方向变化的图像的2D渲染；FIG1A shows a 2D rendering of an image based on changes in the orientation of an observer relative to a display;
图1B示出了由于左眼和右眼的水平分离视差(parallax)而产生的具有双眼视差(disparity)的图像的2D渲染；FIG. 1B shows a 2D rendering of an image with binocular disparity due to horizontal separation parallax of the left and right eyes;
图2A是人类眼球结构的截面图的图示；FIG2A is an illustration of a cross-sectional view of the structure of a human eyeball;
图2B是关于视杆细胞和视锥细胞的密度与中央凹的位置的图；FIG2B is a graph of the density of rods and cones versus the location of the fovea;
图3是观察者视场的顶视图；FIG3 is a top view of the observer's field of view;
图4A是识别使用相机或其他捕获装置捕获的场景的平面的侧视图图示；FIG4A is a side view illustration of a plane identifying a scene captured using a camera or other capture device;
图4B A是利用图8G所示的捕获装置捕获的图4A中的场景的两个图像的示例性实施方式的前视图；FIG4B A is a front view of an exemplary embodiment of two images of the scene in FIG4A captured using the capture device shown in FIG8G;
图5是识别场景的平面以及按图4的比例的舒适圆的顶视图；FIG5 is a top view of the plane of the recognition scene and the comfort circle according to the scale of FIG4;
图6是本公开的计算机系统的框图；FIG6 is a block diagram of a computer system of the present disclosure;
图7是由图1中的计算机系统实现的通信系统的框图；FIG7 is a block diagram of a communication system implemented by the computer system in FIG1;
图8A是计算装置的示例性实施方式的图，该计算装置的四个图像捕获装置定位在眼内或瞳孔间距离宽度(即普通人的瞳孔之间的距离)内竖直线性串联；8A is a diagram of an exemplary embodiment of a computing device with four image capture devices positioned vertically linearly in series within the intraocular or interpupillary distance width (i.e., the distance between the pupils of an average person);
图8B是计算装置的示例性实施方式的图，该计算装置的四个图像捕获装置水平在眼内或瞳孔间距离宽度(即普通人的瞳孔之间的距离)内线性串联定位；FIG8B is a diagram of an exemplary embodiment of a computing device with four image capture devices positioned linearly in series horizontally within the intraocular or interpupillary distance width (i.e., the distance between the pupils of an average person);
图8C是图8A和图8B的线性地串联的四个图像捕获装置的示例性实施方式的分解图；FIG8C is an exploded view of an exemplary embodiment of four image capture devices linearly connected in series of FIGS. 8A and 8B ;
图8D是图8A和图8B的线性地串联的四个图像捕获装置的示例性实施方式的截面图；8D is a cross-sectional view of an exemplary embodiment of four image capture devices of FIGS. 8A and 8B connected linearly in series;
图8E是在眼内或瞳孔间距离宽度(即普通人的瞳孔之间的距离)内线性串联的三个图像捕获装置的示例性实施方式的分解图；FIG8E is an exploded view of an exemplary embodiment of three image capture devices connected linearly in series within the intraocular or interpupillary distance width (i.e., the distance between the pupils of an average person);
图8F是图8E的线性串联的三个图像捕获装置的示例性实施方式的截面图；FIG8F is a cross-sectional view of an exemplary embodiment of three image capture devices in linear series of FIG8E;
图8G是在眼内或瞳孔间距离宽度(即普通人的瞳孔之间的距离)内线性串联的两个图像捕获装置的示例性实施方式的分解图；FIG8G is an exploded view of an exemplary embodiment of two image capture devices connected linearly in series within the intraocular or interpupillary distance width (i.e., the distance between the pupils of an average person);
图8H是图8G的线性串联的两个图像捕获装置的示例性实施方式的截面图；FIG8H is a cross-sectional view of an exemplary embodiment of two image capture devices in linear series of FIG8G;
图9是人眼间隔眼内或瞳孔间距离宽度(即普通人的瞳孔之间的距离)的示例性实施方式的图；FIG. 9 is a diagram of an exemplary embodiment of the width of the intraocular or interpupillary distance of a human eye (i.e., the distance between the pupils of an average person);
图10是识别成比例的场景的平面和舒适圆的顶视图，其中直角三角形限定了捕获装置在晶状体平面上的定位；FIG10 is a top view of a plane and comfort circle identifying a scaled scene, wherein a right triangle defines the positioning of the capture device on the lens plane;
图10A是标识了用于计算图10的舒适圆半径的直角三角形的示例性实施方式的顶视图；FIG10A is a top view of an exemplary embodiment identifying a right triangle used to calculate the comfort circle radius of FIG10;
图10B是标识了用于计算捕获装置在图10的晶状体平面上的线性定位的直角三角形的示例性实施方式的顶视图；FIG10B is a top view of an exemplary embodiment identifying a right triangle for calculating the linear positioning of the capture device on the lens plane of FIG10;
图10C是标识了用于计算图10的背板的最佳距离的直角三角形的示例性实施方式的顶视图；FIG. 10C is a top view of an exemplary embodiment identifying a right triangle for calculating an optimal distance for the backplane of FIG. 10 ;
图11是诸如根据本公开的选定实施方式的图11A中的两个图像(帧)之间的点的几何移位的示例性实施方式的图示；FIG. 11 is an illustration of an exemplary embodiment of a geometric shift of points between two images (frames), such as in FIG. 11A , in accordance with selected embodiments of the present disclosure;
图11A是利用图8A至图8F中所示的捕获装置捕获的、并且关于关键主体点进行对准的场景的四个图像的示例性实施方式的前顶视图；FIG11A is a front top view of an exemplary embodiment of four images of a scene captured using the capture device shown in FIGS. 8A-8F and aligned with respect to key subject points;
图11B是利用图8A至图8F中所示的捕获装置捕获的、并且关于关键主体点进行对准的场景的四个图像的示例性实施方式的正视图；FIG. 11B is a front view of an exemplary embodiment of four images of a scene captured using the capture device shown in FIGS. 8A-8F and aligned with respect to key subject points;
图12是生成利用图8A至图8H中所示的捕获装置捕获的多维图像序列的方法的流程图的示例性实施方式；12 is an exemplary embodiment of a flow chart of a method for generating a multi-dimensional image sequence captured using the capture device shown in FIGS. 8A to 8H ;
图13是具有用于选择计算机系统的摄影选项的用户互动内容的显示器的示例性实施方式；FIG13 is an exemplary embodiment of a display with user interactive content for selecting a photography option of a computer system;
图14A是识别利用图8A至图8F中所示的捕获装置捕获的两个帧的顶视图，其示出了如图11B中所示对准的关键主体和两个帧之间的近平面物体偏移；FIG14A is a top view identifying two frames captured using the capture device shown in FIGS. 8A to 8F showing the key subject aligned as shown in FIG11B and the near plane object offset between the two frames;
图14B是通过图14A的两个帧之间的物体偏移的左眼和右眼虚拟深度的示例性实施方式的顶视图；FIG14B is a top view of an exemplary embodiment of left-eye and right-eye virtual depth by object offset between the two frames of FIG14A;
图15A是根据本公开的选定实施方式的显示器堆的示例性实施方式的截面图；15A is a cross-sectional view of an exemplary embodiment of a display stack according to selected embodiments of the present disclosure;
图15B是根据本公开的选定实施方式的弧形或曲线形透镜的示例性实施方式的截面图，追踪穿过其中的RGB光；15B is a cross-sectional view of an exemplary embodiment of an arc or curved lens tracking RGB light therethrough in accordance with selected embodiments of the present disclosure;
图15C是根据本公开的选定实施方式的梯形透镜的原型实施方式的截面图，追踪RGB光穿过其中；|15C is a cross-sectional view of a prototype implementation of a trapezoidal lens tracing RGB light therethrough in accordance with selected embodiments of the present disclosure; |
图15D是根据本公开的选定实施方式的圆顶形透镜的示例性实施方式的截面图，追踪穿过其中的RGB光；15D is a cross-sectional view of an exemplary embodiment of a dome-shaped lens tracking RGB light therethrough in accordance with selected embodiments of the present disclosure;
图16A是诸如根据本公开的选定实施方式的图8A中的图像(帧)的像素间相处理(pixel interphase processing)的示例性实施方式的图示；FIG. 16A is a diagram of an exemplary implementation of pixel interphase processing of an image (frame) such as in FIG. 8A in accordance with selected implementations of the present disclosure;
图16B是运行应用的计算机系统的显示器的示例性实施方式的俯视图；FIG16B is a top view of an exemplary embodiment of a display of a computer system running an application;
图17是观看显示器上的多维数字图像的示例性实施方式的俯视图，其中图像在舒适圆内、靠近同视点弧或点、在Panum区域内并且从观看距离观看；FIG17 is a top view of an exemplary embodiment of viewing a multi-dimensional digital image on a display, wherein the image is within a comfort circle, near a synoptophore arc or point, within the Panum region, and viewed from a viewing distance;
图18是利用具有图8A至图8H中所示的图像捕获装置的计算装置来生成不可替代令牌(NFT)的方法的流程图的示例性实施方式；FIG. 18 is an exemplary embodiment of a flow chart of a method for generating a non-fungible token (NFT) using a computing device having the image capture device shown in FIGS. 8A-8H ;
图19是利用具有图8A至图8H中所示的图像捕获装置捕获装置的计算装置来选择DIGY序列并向其添加音频文件的方法的流程图的示例性实施方式；以及以及FIG. 19 is an exemplary embodiment of a flowchart of a method of selecting a DIGY sequence and adding an audio file thereto using a computing device having the image capture device shown in FIGS. 8A-8H ; and
图20是利用具有图8A至图8H中所示的图像捕获装置捕获装置的计算装置来生成DIGY并向其添加音频文件的方法的流程图的示例性实施方式。20 is an exemplary embodiment of a flowchart of a method of generating a DIGY and adding an audio file thereto using a computing device having the image capture device shown in FIGS. 8A-8H .
需要指出的是，所呈现的附图仅仅旨在出于说明的目的，因此，它们既不希望也不旨在将公开内容限制在所示出的任何或所有确切的构造细节上，除非它们可能被视为对所要求保护的公开内容至关重要。It is noted that the drawings presented are intended for illustrative purposes only and, therefore, they are neither intended nor limited to any or all of the exact construction details shown, unless they may be deemed essential to the claimed disclosure.
具体实施方式Detailed ways
在描述本公开的示例性实施方式时，如图所示，为清晰起见，采用了特定的术语。然而，本公开并不旨在局限于如此选择的具体术语，应理解为每个具体元素包括以类似方式操作以完成类似功能的所有技术等同物。然而，所要求保护的发明可以以许多不同的形式体现出来，并且不应解释为仅限于本文所阐述的实施方式。本文所列举的示例是非限制性示例，并且只是在其他可能的示例中的示例。When describing the exemplary embodiments of the present disclosure, as shown in the figure, specific terms are used for clarity. However, the present disclosure is not intended to be limited to the specific terms so selected, and it should be understood that each specific element includes all technical equivalents that operate in a similar manner to perform similar functions. However, the claimed invention can be embodied in many different forms and should not be interpreted as being limited to the embodiments set forth herein. The examples listed herein are non-limiting examples and are only examples among other possible examples.
对深度的感知是基于各种线索的，双眼视差(binocular disparity)和运动视差(motion parallax)通常比图像线索提供更精确的深度信息。双眼视差和运动视差为深度感知提供了两个独立的定量线索。双眼视差指的是一个点在3D空间中的两个视网膜图像投影之间的位置差异。如图1A和图1B所示，在图像场景110中观看物体102时获得的鲁健的深度概念表明：大脑可以仅根据双眼视差线索来计算深度。在双眼视觉中，双眼单视界112是空间中与固定点114具有相同视差的点的位置。位于通过固定点114的水平线上的物体会产生单个图像，而距离此线一定距离的物体会产生两个图像116、118。The perception of depth is based on various cues, and binocular disparity and motion parallax generally provide more accurate depth information than image cues. Binocular disparity and motion parallax provide two independent quantitative cues for depth perception. Binocular disparity refers to the position difference between two retinal image projections of a point in 3D space. As shown in Figures 1A and 1B, Lu Jian's concept of depth obtained when viewing an object 102 in an image scene 110 shows that the brain can calculate depth based solely on binocular disparity cues. In binocular vision, the binocular single vision 112 is the position of a point in space that has the same disparity as a fixed point 114. Objects located on a horizontal line passing through the fixed point 114 will produce a single image, while objects at a certain distance from this line will produce two images 116 and 118.
经典的运动视差取决于两个眼睛功能。一个是眼睛对运动的跟踪(眼球移动以将运动固定在单个点上)，以及第二个是光滑的运动差异，导致视差或双眼视差。经典的运动视差是指观察者是静止的而观察者周围的场景是平移的情况，或者相反，场景是静止的而观察者在场景中平移的情况。Classical motion parallax depends on two eye functions. One is the tracking of motion by the eye (eye movement to fix the motion on a single point), and the second is the smooth difference in motion, resulting in parallax or binocular disparity. Classical motion parallax refers to the situation where the observer is stationary and the scene around the observer is translating, or vice versa, the scene is stationary and the observer is translating within the scene.
通过使用从略微不同的角度获得的同一物体102的两个图像116、118，可以以高度准确性三角测量与物体102的距离。每只眼睛观看由左眼104和右眼106看到的物体102的角度略有不同。这是因为眼睛的水平分离视差而发生的。如果一个物体很远，该图像110落在两个视网膜上的视差108会很小。如果物体很靠近或很接近，则该图像110落在两个视网膜上的视差108将很大。By using two images 116, 118 of the same object 102 obtained from slightly different angles, the distance to the object 102 can be triangulated with a high degree of accuracy. Each eye sees the object 102 seen by the left eye 104 and the right eye 106 at a slightly different angle. This occurs because of the horizontal separation parallax of the eyes. If an object is far away, the parallax 108 of the image 110 falling on the two retinas will be small. If the object is close or very close, the parallax 108 of the image 110 falling on the two retinas will be large.
运动视差120指的是由于观察者104的平移而导致的相对图像运动(在不同深度处的物体之间)。与双眼深度线索和图画深度线索分开，运动视差120也可以提供精确的深度感知，条件是它伴随着指定眼睛方向相对于视觉场景110的变化的辅助信号。如图所示，随着眼睛方向104的变化，物体102相对于背景的明显相对运动给出了关于其相对距离的提示。如果物体102很远，物体102看起来是静止的。如果物体102很靠近或很接近，物体102看起来会更快地移动。Motion parallax 120 refers to relative image motion (between objects at different depths) due to translation of the observer 104. Separate from binocular depth cues and pictorial depth cues, motion parallax 120 can also provide accurate depth perception if it is accompanied by an auxiliary signal that specifies changes in eye direction relative to the visual scene 110. As shown, the apparent relative motion of the object 102 relative to the background as the eye direction 104 changes gives a clue as to its relative distance. If the object 102 is far away, the object 102 appears to be stationary. If the object 102 is close or very close, the object 102 appears to move faster.
为了在近处看到物体102，并将两个视网膜上的图像融合成一个物体，两只眼睛104、106的光轴会聚在物体102上。改变眼睛晶状体的焦距以便将聚焦的图像放在视网膜的中央凹处的肌肉动作被称为调节。肌肉动作和相邻深度的不聚焦都为大脑提供了可以用于感知深度的额外的信息。图像清晰度是一个模糊的深度线索。然而，通过改变聚焦平面(看起来比物体102更近和/或更远)，可以解决模糊性问题。In order to see the object 102 at close range and fuse the images on the two retinas into one object, the optical axes of the two eyes 104, 106 converge on the object 102. The muscle action that changes the focal length of the eye lens so that the focused image is placed at the fovea of the retina is called accommodation. Both the muscle action and the unfocusedness of adjacent depths provide the brain with additional information that can be used to perceive depth. Image sharpness is an ambiguous depth cue. However, by changing the plane of focus (appearing closer and/or farther than the object 102), the ambiguity can be resolved.
图2A和图2B分别示出了眼睛200的解剖结构以及视杆细胞和视锥细胞的分布图的图形表示。中央凹202负责敏锐的中央视觉(也被称为中央凹视觉)，这在视觉细节是最重要的情况下是必要的。中央凹202是视网膜内表面205的凹陷部，约1.5mm宽，并且完全由专门用于最大视力的视锥细胞204组成。视杆细胞206是接收灰度信息的低强度受体，并且对周边视觉很重要，而视锥细胞204是接收色觉信息的高强度受体。参照图2B，可以更清楚地理解中央凹202的重要性，图2B示出了视锥细胞204和视杆细胞206在眼睛200中的分布。如图所示，提供最高视觉敏锐度的大部分视锥细胞204位于中央凹202中心周围1.5°的角度内。2A and 2B show graphical representations of the anatomy of the eye 200 and the distribution of rods and cones, respectively. The fovea 202 is responsible for sharp central vision (also known as foveal vision), which is necessary when visual detail is of utmost importance. The fovea 202 is a depression in the inner surface 205 of the retina, approximately 1.5 mm wide, and is composed entirely of cones 204, which are specialized for maximum vision. Rods 206 are low-intensity receptors that receive grayscale information and are important for peripheral vision, while cones 204 are high-intensity receptors that receive color vision information. The importance of the fovea 202 can be more clearly understood with reference to FIG. 2B , which shows the distribution of cones 204 and rods 206 in the eye 200. As shown, most of the cones 204, which provide the highest visual acuity, are located within an angle of 1.5° around the center of the fovea 202.
参照图2B，可以更清楚地了解中央凹202的重要性，图2B示出了眼睛200中视锥细胞204和视杆细胞206的分布。如图所示，提供最高视觉敏锐度的大部分视锥细胞204位于围绕中央凹202中心的1.5°的角度内。2B, which shows the distribution of cones 204 and rods 206 in the eye 200. As shown, most of the cones 204, which provide the highest visual acuity, are located within an angle of 1.5° around the center of the fovea 202.
图3示出了人类视觉系统(HVS)的典型视场300。如图所示，中央凹202只看到视场302的中央1.5°(度)，其中优选的视场304位于中央凹202中心的±15°(度)之内。因此，将物体聚焦在中央凹上，取决于物体102的线性尺寸、观察角度和观察距离。在近处观看的大物体102将有一个大的视角落在中央凹视觉之外，而在远处观看的小物体102将有一个小的视角落在中央凹视觉之内。落在中央凹视觉内的物体102将在头脑中产生高的视觉锐度。然而，在自然观看条件下，观看者不只是被动地感知。相反，他们通过在不同观看距离处在物体之间移位眼睛的固定点和焦点来动态地扫描视觉场景110。在这样做的时候，眼球运动过程中的调节和辐辏(左眼104的视线和右眼106的视线之间的角度)必须同步移位，以将新的物体置于每个视网膜中心的清晰焦点上。因此，自然将调节和辐辏反射性地联系在一起，这样，一个过程的变化会自动地驱动另一个过程的相应变化。FIG. 3 shows a typical field of view 300 of the human visual system (HVS). As shown, the fovea 202 only sees the central 1.5° (degrees) of the field of view 302, with the preferred field of view 304 being within ±15° (degrees) of the center of the fovea 202. Therefore, focusing an object on the fovea depends on the linear size of the object 102, the viewing angle, and the viewing distance. Large objects 102 viewed at close range will have a large viewing angle falling outside the foveal vision, while small objects 102 viewed at a distance will have a small viewing angle falling within the foveal vision. Objects 102 that fall within the foveal vision will produce high visual acuity in the mind. However, under natural viewing conditions, viewers do not just perceive passively. Instead, they dynamically scan the visual scene 110 by shifting the fixation point and focus of the eyes between objects at different viewing distances. In doing so, the accommodation and convergence (the angle between the line of sight of the left eye 104 and the line of sight of the right eye 106) during eye movement must be shifted synchronously to place the new object in sharp focus at the center of each retina. Thus, nature links regulation and convergence reflexively, so that changes in one process automatically drive corresponding changes in the other.
图4A示出了将由相机或数字图像捕获装置—诸如图像捕获模块830—捕获的场景S的典型视图。场景S可以包括四个平面，被限定为：(1)透镜框架被限定为通过记录装置或相机中的图像透镜或传感器(图像捕获模块830)的平面，(2)关键主体平面KSP可以是通过场景中的传感器焦点(这里是场景中的对(couple)，场景S的关键主体KS)的平面，(3)近平面NP可以是通过与透镜平面最近的焦点的平面(前景中的灌木B)，以及(4)远平面FP，其是通过最远焦点的平面(后景中的树T)。与图像捕获模块830的相对距离用N、Ks、B表示。场景S的景深是由近平面NP和远平面FP之间的距离定义的。FIG. 4A shows a typical view of a scene S to be captured by a camera or digital image capture device, such as image capture module 830. Scene S may include four planes, defined as: (1) a lens frame is defined as a plane through an image lens or sensor in a recording device or camera (image capture module 830), (2) a key subject plane KSP may be a plane through sensor focal points in the scene (here a couple in the scene, key subjects KS of scene S), (3) a near plane NP may be a plane through focal points closest to the lens plane (bush B in the foreground), and (4) a far plane FP, which is a plane through the farthest focal points (trees T in the background). Relative distances from image capture module 830 are denoted by N, Ks, B. The depth of field of scene S is defined by the distance between near plane NP and far plane FP.
如上所述，立体图像的深度感根据相机与关键主体之间的距离—称为图像捕获距离或KS—而变化。深度感也受到辐辏角和相机捕获每个连续图像之间的眼内距离的控制，这影响了双眼视差。As mentioned above, the depth perception of a stereo image varies depending on the distance between the camera and the key subject—known as the image capture distance, or KS. Depth perception is also controlled by the angle of convergence and the intraocular distance between each successive image captured by the camera, which affects binocular parallax.
在摄影中，“弥散圆”定义了场景S中被捕获到的聚焦的区域。因此，近平面NP、关键主体平面KSP和远平面FP都聚焦。这个圆外的区域是模糊的。In photography, a "circle of confusion" defines the area of the scene S that is captured in focus. Thus, the near plane NP, the key subject plane KSP, and the far plane FP are all in focus. The area outside this circle is blurred.
图4B示出了要由相机或数字图像捕获装置，诸如图像捕获模块830，更具体地是图8G所示的图像捕获模块830捕获的场景S的典型视图。两个图像捕获装置831和832或任何其他选择的对831、832、833、834可以用于捕获场景S的多个数字图像作为场景S的左图像810L和右图像810R，如图8A所示(多个数字图像)。替代性地，计算机系统10通过图像操纵应用和显示器208可以配置成使用户U能够选择或标识图像捕获装置831(1)、832(2)、833(3)或834(4)中的两个图像捕获装置以捕获场景S的两个数字图像作为场景S的左图像810L和右图像810R。用户U可以点击或与选择框812进行其他识别交互以选择或标识场景S的源图像、左图像810L和右图像810R中的关键主体KS，如图4B所示。FIG4B shows a typical view of a scene S to be captured by a camera or digital image capture device, such as an image capture module 830, more specifically the image capture module 830 shown in FIG8G. Two image capture devices 831 and 832 or any other selected pair 831, 832, 833, 834 can be used to capture multiple digital images of the scene S as a left image 810L and a right image 810R of the scene S, as shown in FIG8A (multiple digital images). Alternatively, the computer system 10, through the image manipulation application and the display 208, can be configured to enable the user U to select or identify two of the image capture devices 831 (1), 832 (2), 833 (3), or 834 (4) to capture two digital images of the scene S as a left image 810L and a right image 810R of the scene S. The user U can click or otherwise identify the selection box 812 to select or identify the key subject KS in the source image, the left image 810L, and the right image 810R of the scene S, as shown in FIG4B.
图5示出了按图4.1和图3.1的比例的舒适圆(CoC)。将舒适圆(CoC)定义为通过使圆的直径从镜头平面—图像捕获模块830—上的中心点沿关键主体平面KSP的垂直线(按图4的比例)通过(其中宽度由图3的30度径向决定)而形成的圆。(R是舒适圆(CoC)的半径)。FIG5 shows a comfort circle (CoC) to the scale of FIG4.1 and FIG3.1. The comfort circle (CoC) is defined as the circle formed by passing the diameter of the circle from the center point on the lens plane—the image capture module 830—along a perpendicular line (to the scale of FIG4) to the key subject plane KSP, with the width determined by the 30 degree radial of FIG3. (R is the radius of the comfort circle (CoC)).
传统的立体显示器迫使观看者试图将这些过程解耦，因为当他们必须动态地改变辐辏角以观看不同立体距离处的对象时，他们必须将调节保持在一个固定的距离，否则整个显示器将滑出焦点。在观看这种显示器时，这种解耦会产生眼睛疲劳，并损害图像质量。Conventional stereoscopic displays force viewers to try to decouple these processes because while they must dynamically change the angle of convergence to view objects at different stereoscopic distances, they must maintain accommodation at a fixed distance or the entire display will slide out of focus. This decoupling can produce eye fatigue and impair image quality when viewing such displays.
为了理解本公开的内容，需要对某些变量进行定义。物场是指正在构成的整个图像。“关键主体点”被定义为场景会聚处的点，即景深中始终保持聚焦且在关键主体点上没有视差的点。前景点和背景点分别是距观看者的最近点和最远点。景深是在物场内创建的深度或距离(描绘的前景到背景的距离)。主轴是垂直于场景、通过关键主体点的线。视差或双眼视差是指在关键主体对准后任何点在第一张和最后一张图像中的位置差异。在数字构图中，帧之间的关键主体点与主轴的移位始终保持为与主轴的整数像素数。总视差是最近帧的关键主体点与主轴的移位的绝对值与最远帧的关键主体点与主轴的移位的绝对值之和。In order to understand the content of this disclosure, certain variables need to be defined. The object field refers to the entire image being composed. The "key subject point" is defined as the point where the scene converges, that is, the point in the depth of field that always remains in focus and has no parallax on the key subject point. The foreground point and the background point are the closest and farthest points to the viewer, respectively. Depth of field is the depth or distance created within the object field (the distance of the depicted foreground to the background). The main axis is a line perpendicular to the scene that passes through the key subject point. Parallax or binocular disparity refers to the difference in position of any point in the first and last image after the key subject is aligned. In digital composition, the displacement of the key subject point from the main axis between frames is always maintained as an integer number of pixels from the main axis. The total parallax is the sum of the absolute value of the displacement of the key subject point from the main axis of the closest frame and the absolute value of the displacement of the key subject point from the main axis of the farthest frame.
此处在捕获图像时，申请人指的是景深或弥散圆，而在观看装置上观看图像时指的是舒适圆。Applicants refer herein to the depth of field or circle of confusion when capturing an image and to the circle of comfort when viewing the image on a viewing device.
美国专利9,992,473、美国专利10,033,990和美国专利10,178,247的全部内容通过引用并入本文。The entire contents of US Patent 9,992,473, US Patent 10,033,990, and US Patent 10,178,247 are incorporated herein by reference.
利用运动视差创建深度感知是已知的。然而，为了最大化深度，同时保持令人愉悦的观看体验，引入了一种系统方法。该系统将人类视觉系统的因素与图像捕获程序相结合，以在任何2D观看装置上产生真实的深度体验。Creating depth perception using motion parallax is known. However, in order to maximize depth while maintaining a pleasing viewing experience, a systematic approach has been introduced. This system combines factors of the human visual system with image capture procedures to produce a realistic depth experience on any 2D viewing device.
该技术引入了舒适圆(CoC)，舒适圆规定了图像捕获系统相对于场景S的位置。舒适圆(CoC)相对于关键主体KS(会聚点，焦点)设置了最佳的近平面NP和远平面FP，即控制了场景S的视差。The technology introduces a comfort circle (CoC), which specifies the position of the image capture system relative to the scene S. The comfort circle (CoC) sets the optimal near plane NP and far plane FP relative to the key subject KS (convergence point, focus), that is, the parallax of the scene S is controlled.
该系统的开发使任何捕获装置，诸如iPhone、相机或摄像机都可以用来捕获场景。类似地，捕获的图像可以在任何数字输出装置诸如智能手机、平板电脑、监控器、电视、笔记本电脑或电脑屏幕上进行组合和观看。The system was developed so that any capture device, such as an iPhone, camera or camcorder can be used to capture the scene. Similarly, the captured images can be combined and viewed on any digital output device such as a smartphone, tablet, monitor, TV, laptop or computer screen.
正如本领域的技术人员所理解的那样，本公开可以体现为一种方法、数据处理系统或计算机程序产品。因此，本公开的内容可以采取完全是硬件的实施方式、完全软件的实施方式、或结合软件和硬件方面的实施方式。此外，本公开可以采取计算机可读存储介质上的计算机程序产品的形式，该计算机可读存储介质具有体现在该介质中的计算机可读程序代码装置。可以利用任何合适的计算机可读介质，包括硬盘、ROM、RAM、CD-ROM、电、光、磁存储装置等。As will be appreciated by those skilled in the art, the present disclosure may be embodied as a method, a data processing system, or a computer program product. Thus, the content of the present disclosure may take the form of an entirely hardware implementation, an entirely software implementation, or an implementation combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product on a computer-readable storage medium having a computer-readable program code device embodied in the medium. Any suitable computer-readable medium may be utilized, including hard disks, ROMs, RAMs, CD-ROMs, electrical, optical, magnetic storage devices, and the like.
下面参照根据本公开的实施方式的方法、设备(系统)和计算机程序产品的流程图图示来描述本公开。将理解的是，流程图图示中的每个块或步骤、以及流程图图示中的块或步骤的组合可以通过计算机程序指令或操作来实现。这些计算机程序指令或操作可以加载到通用计算机、专用计算机或其他可编程数据处理设备上，以生产一台机器，从而使在计算机或其他可编程数据处理设备上执行的指令或操作创建用于实现流程图块/步骤中规定功能的装置。The present disclosure is described below with reference to flowchart illustrations of methods, devices (systems) and computer program products according to embodiments of the present disclosure. It will be understood that each block or step in the flowchart illustration, and the combination of blocks or steps in the flowchart illustration, can be implemented by computer program instructions or operations. These computer program instructions or operations can be loaded onto a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that the instructions or operations executed on the computer or other programmable data processing device create a device for implementing the functions specified in the flowchart blocks/steps.
这些计算机程序指令或操作也可以存储在计算机可使用的存储器中，该存储器可以指导计算机或其他可编程数据处理设备以特定方式运行，从而使存储在计算机可使用的存储器中的指令或操作产生一个制造品，该制造品包括实现一个或更多个流程图块/步骤中指定的功能的指令装置。计算机程序指令或操作也可以加载到计算机或其他可编程数据处理设备(处理器)上，以使一系列操作步骤在计算机或其他可编程设备(处理器)上执行，以产生计算机实现的过程，从而使在计算机或其他可编程设备(处理器)上执行的指令或操作提供用于实现一个或更多个流程图块/步骤中指定的功能的步骤。These computer program instructions or operations may also be stored in a computer-usable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions or operations stored in the computer-usable memory produce an article of manufacture that includes instruction means for implementing the functions specified in one or more flowchart blocks/steps. The computer program instructions or operations may also be loaded onto a computer or other programmable data processing device (processor) so that a series of operating steps are executed on the computer or other programmable device (processor) to produce a computer-implemented process, so that the instructions or operations executed on the computer or other programmable device (processor) provide steps for implementing the functions specified in one or more flowchart blocks/steps.
因此，流程图图示的块或步骤支持用于执行指定功能的装置的组合、用于执行指定功能的步骤的组合、以及用于执行指定功能的程序指令装置。还应理解的是，流程图图示中的每个块或步骤、以及流程图图示中的块或步骤的组合可以由基于专用硬件的计算机系统来实现，该系统执行指定的功能或步骤、或专用硬件和计算机指令或操作的组合。Therefore, the blocks or steps of the flowchart illustration support a combination of means for performing the specified functions, a combination of steps for performing the specified functions, and a program instruction device for performing the specified functions. It should also be understood that each block or step in the flowchart illustration, and the combination of blocks or steps in the flowchart illustration can be implemented by a computer system based on special-purpose hardware, which performs the specified functions or steps, or a combination of special-purpose hardware and computer instructions or operations.
用于实现本公开的计算机编程可以以各种编程语言、数据库语言等编写。然而，可以理解的是，在不背离本公开的精神和意图的情况下，可以利用其他源或面向对象的编程语言、以及其他常规编程语言。Computer programming for implementing the present disclosure can be written in various programming languages, database languages, etc. However, it will be appreciated that other source or object-oriented programming languages, as well as other conventional programming languages, may be utilized without departing from the spirit and intent of the present disclosure.
现在参照图6，图6示出了计算机系统10的框图，计算机系统10提供了用于实现本公开的实施方式的合适的环境。图6中所示的计算机架构分为两部分—主板600和输入/输出(I/O)装置620。主板600优选地包括通过总线10互连的：用于执行指令的子系统或处理器，诸如中央处理单元(CPU)602；存储器装置，诸如随机存取存储器(RAM)604；输入/输出(I/O)控制器608；以及存储器装置，诸如只读存储器(ROM)606，也被称为固件。基本输入输出系统(BIOS)包含有助于在计算机子系统内各元素之间传输信息的基本例程，基本输入输出系统(BIOS)优选地存储在ROM 606中，或可操作地设置在RAM 604中。计算机系统10还优选地包括：I/O装置620，诸如用于存储操作系统626并通过应用624执行指令的主存储装置634；以及用于视觉输出的显示器628；以及其他适当的I/O装置632。主存储装置634优选地通过连接到总线610的主存储控制器(表示为608)与CPU 602连接。网络适配器630允许计算机系统通过通信装置或任何其他能够通过通信链路传输和接收数据的网络适配器发送和接收数据，该通信链路可以是有线、光学或无线数据通路。在此认识到，中央处理单元(CPU)602执行存储在ROM 606或RAM 604中的指令、操作或命令。Referring now to FIG. 6 , FIG. 6 shows a block diagram of a computer system 10 that provides a suitable environment for implementing embodiments of the present disclosure. The computer architecture shown in FIG. 6 is divided into two parts—a motherboard 600 and input/output (I/O) devices 620. The motherboard 600 preferably includes: a subsystem or processor for executing instructions, such as a central processing unit (CPU) 602; a memory device, such as a random access memory (RAM) 604; an input/output (I/O) controller 608; and a memory device, such as a read-only memory (ROM) 606, also known as firmware, interconnected by a bus 10. The basic input-output system (BIOS) contains basic routines that help transfer information between elements within the computer subsystem, and the basic input-output system (BIOS) is preferably stored in the ROM 606 or operably disposed in the RAM 604. The computer system 10 also preferably includes: an I/O device 620, such as a main storage device 634 for storing an operating system 626 and executing instructions through an application 624; and a display 628 for visual output; and other appropriate I/O devices 632. The main storage device 634 is preferably connected to the CPU 602 through a main storage controller (represented as 608) connected to the bus 610. The network adapter 630 allows the computer system to send and receive data through a communication device or any other network adapter capable of transmitting and receiving data through a communication link, which can be a wired, optical or wireless data path. It is recognized that the central processing unit (CPU) 602 executes instructions, operations or commands stored in the ROM 606 or RAM 604.
在此设想，计算机系统10可以包括智能装置，诸如智能手机、iPhone、安卓手机(谷歌、三星或其他制造商)、平板电脑、台式机、笔记本电脑、数字图像捕获装置以及其他具有两个或更多个数字图像捕获装置和/或3D显示器608的计算装置(智能装置)。It is contemplated herein that the computer system 10 may include smart devices such as smartphones, iPhones, Android phones (Google, Samsung, or other manufacturers), tablets, desktops, laptops, digital image capture devices, and other computing devices (smart devices) with two or more digital image capture devices and/or a 3D display 608.
在此进一步设想，显示器608可以被配置成能够展开成更大的显示表面区域的可折叠显示器或可多次折叠显示器。It is further contemplated that the display 608 may be configured as a foldable display or a multiply foldable display capable of unfolding to a larger display surface area.
许多其他装置或子系统或其他I/O装置632可以以类似的方式连接，包括但不限于诸如麦克风、扬声器、闪存驱动器、CD-ROM播放器、DVD播放器、打印机、主存储装置634诸如硬盘、和/或调制解调器等的装置，这些装置各自通过I/O适配器连接。另外，虽然是优选的，但不一定要存在图6中所示的所有装置才能实践本公开的内容，如下文所讨论的。此外，这些装置和子系统可以以与图6中所示不同的配置互连，也可以基于光学或门阵列、或能够响应和执行指令或操作的这些元素的某种组合互连。如图6中所示的计算机系统的操作在本领域中是很容易知道的，并且在本申请中没有进一步详细讨论，以便不使本讨论过于复杂。Many other devices or subsystems or other I/O devices 632 can be connected in a similar manner, including but not limited to devices such as microphones, speakers, flash drives, CD-ROM players, DVD players, printers, primary storage devices 634 such as hard disks and/or modems, etc., which are each connected through an I/O adapter. In addition, although it is preferred, it is not necessary to have all the devices shown in Figure 6 to practice the content of the present disclosure, as discussed below. In addition, these devices and subsystems can be interconnected in configurations different from those shown in Figure 6, or can be interconnected based on optics or gate arrays or some combination of these elements that can respond and execute instructions or operations. The operation of a computer system as shown in Figure 6 is well known in the art and is not further discussed in detail in this application so as not to make this discussion too complicated.
现在参照图7，图2示出了描绘了示例性通信系统700的图，其中可以实现与本公开一致的概念。图7的通信系统700内的每个元素的示例在以上参照图6进行了广泛描述。特别地，服务器系统760和用户系统720具有类似于图6的计算机系统10的属性，并示出了计算机系统10的一种可能的实现方式。通信系统700优选地包括一个或更多个用户系统720、722、724(在此设想计算机系统10可以包括智能装置，诸如智能手机、iPhone、安卓手机(谷歌、三星或其他制造商)、平板电脑、台式机、笔记本电脑、相机和其他具有显示器628的计算装置(智能装置))、一个或更多个服务器系统760和网络750，该网络可以是例如因特网、公共网络、私人网络或云。用户系统720至724每个优选地包括耦接至处理器的计算机可读介质，诸如随机存取存储器604、606。处理器—CPU 702—执行存储在存储器604、606中的程序指令或操作(应用软件624)。通信系统700通常包括一个或更多个用户系统720。例如，用户系统720可以包括一个或更多个通用计算机(例如，个人计算机)、一个或更多个专用计算机(例如，专门编程为彼此和/或服务器系统760进行通信的装置)、工作站、服务器、装置、数字助理或“智能”移动电话或寻呼机、数字相机、部件、其他设备、或能够响应于并执行指令或操作的这些元素的一些组合。Referring now to FIG. 7 , FIG. 2 shows a diagram depicting an exemplary communication system 700 in which concepts consistent with the present disclosure may be implemented. Examples of each element within the communication system 700 of FIG. 7 are extensively described above with reference to FIG. 6 . In particular, the server system 760 and the user system 720 have properties similar to the computer system 10 of FIG. 6 , and illustrate one possible implementation of the computer system 10. The communication system 700 preferably includes one or more user systems 720 , 722 , 724 (it is contemplated that the computer system 10 may include smart devices, such as smartphones, iPhones, Android phones (Google, Samsung or other manufacturers), tablets, desktops, laptops, cameras and other computing devices (smart devices) with displays 628 ), one or more server systems 760 and a network 750 , which may be, for example, the Internet, a public network, a private network or a cloud. Each of the user systems 720 to 724 preferably includes a computer-readable medium coupled to a processor, such as a random access memory 604 , 606 . The processor—CPU 702—executes program instructions or operations (application software 624) stored in memory 604, 606. The communication system 700 typically includes one or more user systems 720. For example, the user systems 720 may include one or more general purpose computers (e.g., personal computers), one or more special purpose computers (e.g., devices specifically programmed to communicate with each other and/or the server system 760), workstations, servers, appliances, digital assistants or "smart" mobile phones or pagers, digital cameras, components, other devices, or some combination of these elements that are capable of responding to and executing instructions or operations.
与用户系统720类似，服务器系统760优选地包括耦接至处理器的计算机可读介质，诸如随机存取存储器604、606。该处理器执行存储在存储器604、606中的程序指令。服务器系统760还可以包括许多附加的外部或内部装置，诸如但不限于，鼠标、CD-ROM、键盘、显示器、存储装置和其他类似于图6的计算机系统10的属性。服务器系统760可以另外地包括二级存储元件，诸如用于存储数据和信息的数据库770。服务器系统760虽然被描述为单一的计算机系统，但其可以实现为计算机处理器的网络。服务器系统760中的存储器604、606包含一个或更多个可执行的步骤、程序、算法或应用624(在图6中示出的)。例如，服务器系统760可以包括网络服务器、信息服务器、应用服务器、一个或更多个通用计算机(例如，个人计算机)、一个或更多个专用计算机(例如，专门编程为相互通信的装置)、工作站或其他设备、或能够响应于和执行指令或操作的这些元素的一些组合。Similar to the user system 720, the server system 760 preferably includes a computer-readable medium, such as random access memory 604, 606, coupled to a processor. The processor executes program instructions stored in the memory 604, 606. The server system 760 may also include many additional external or internal devices, such as, but not limited to, a mouse, a CD-ROM, a keyboard, a display, a storage device, and other attributes similar to the computer system 10 of Figure 6. The server system 760 may additionally include a secondary storage element, such as a database 770 for storing data and information. Although the server system 760 is described as a single computer system, it can be implemented as a network of computer processors. The memory 604, 606 in the server system 760 contains one or more executable steps, programs, algorithms or applications 624 (shown in Figure 6). For example, the server system 760 may include a network server, an information server, an application server, one or more general-purpose computers (e.g., personal computers), one or more special-purpose computers (e.g., devices specially programmed to communicate with each other), a workstation or other device, or some combination of these elements that can respond to and execute instructions or operations.
通信系统700能够通过通信链路740和/或网络750在用户系统720与服务器系统760之间传递和交换数据(包括三维3D图像文件)。通过用户系统720，用户可以优选地通过网络750与其他用户系统720、722、724，以及与其他系统和装置诸如服务器系统760传达数据，以以电子方式传输、存储、打印和/或查看多维数字主图像。通信链路740通常包括在用户系统720与服务器系统760之间进行直接或间接通信的网络750，而不考虑物理分离。网络750的示例包括因特网、云、模拟或数字有线和无线网络、无线电、电视、电缆、卫星和/或任何其他用于承载和/或传输数据或其他信息的传递机制，诸如以电子方式传输、存储、打印和/或查看多维数字主图像。通信链路740可以包括，例如，有线、无线、电缆、光学或卫星通信系统或其他通路。The communication system 700 is capable of transferring and exchanging data (including three-dimensional 3D image files) between the user system 720 and the server system 760 through the communication link 740 and/or the network 750. Through the user system 720, the user can preferably communicate data with other user systems 720, 722, 724 through the network 750, as well as with other systems and devices such as the server system 760 to electronically transmit, store, print and/or view the multi-dimensional digital master image. The communication link 740 generally includes a network 750 for direct or indirect communication between the user system 720 and the server system 760, regardless of physical separation. Examples of the network 750 include the Internet, the cloud, analog or digital wired and wireless networks, radio, television, cable, satellite and/or any other delivery mechanism for carrying and/or transmitting data or other information, such as electronically transmitting, storing, printing and/or viewing the multi-dimensional digital master image. The communication link 740 may include, for example, a wired, wireless, cable, optical or satellite communication system or other pathway.
再次参照图2A、图5和图8A-图8F，为了达到最佳效果和简化数学，捕获场景S的连续图像或帧之间的眼内距离被固定为与人类左眼和右眼的平均分离相匹配，以保持恒定的双眼视差。此外，将与关键主体KS的距离选择成使得捕获的关键主体图像的大小落在观察者的中央凹视觉内，以产生关键主体的高视觉锐度，并保持一个等于或小于15度(15°)的优选的视角的辐辏角。Referring again to FIGS. 2A , 5 , and 8A-8F , for best results and mathematical simplicity, the intraocular distance between successive images or frames of captured scene S is fixed to match the average separation of the human left and right eyes to maintain a constant binocular disparity. Additionally, the distance to the key subject KS is selected so that the size of the captured key subject image falls within the observer's foveal vision to produce high visual acuity of the key subject and maintain a preferred angle of convergence equal to or less than fifteen degrees (15°) of viewing angle.
图8A至图8F公开了一个图像或帧捕获系统，其用于捕获场景S的立体图像(例如，3D序列的2D帧)，诸如图4。此处，在捕获场景S的立体图像(例如，3D序列的2D帧)时，图像捕获距离、图像捕获系统与场景S中的点或平面的距离—诸如关键主体KS和相机的焦距(即放大和缩小)—可以理想地保持不变；然而，如果每个连续的立体图像的捕获装置之间的间距保持不变，则辐辏角将相应地变化。8A to 8F disclose an image or frame capture system for capturing stereoscopic images (e.g., 2D frames of a 3D sequence) of a scene S, such as FIG 4. Here, when capturing stereoscopic images (e.g., 2D frames of a 3D sequence) of a scene S, the image capture distance, the distance of the image capture system from a point or plane in the scene S—such as the key subject KS and the focal length (i.e., zoom in and out) of the camera—can ideally remain constant; however, if the spacing between the capture devices of each consecutive stereoscopic image remains constant, the convergence angle will change accordingly.
现在参照图8A，通过示例而非限制的方式，图示出了计算机系统10，诸如智能装置或便携式智能装置，其具有：背侧810、第一边缘诸如短边缘811和第二边缘诸如长边缘812。背侧810可以包括I/O装置632，诸如图像捕获模块830的示例性实施方式，并且可以包括一个或更多个传感器840，用以测量计算机系统10与图像或场景S中的选定深度(深度)之间的距离。图像捕获模块830可以包括多个或四个数字图像捕获装置831、832、833、834，其中四个数字图像捕获装置关于背侧810或靠近且平行于长边缘812(在眼内或瞳孔间距离宽度IPD(舒适圆关系内的人类视觉系统的瞳孔之间的距离，以优化人类视觉系统的数字多维图像)内竖直线性串联定位。瞳孔间距离宽度IPD优选地是普通人的瞳孔之间的距离，可以具有大约两英寸至两英寸半——2.5英寸(6.35cm)之间、更优选地大约40mm至80mm之间的距离，绝大多数成年人的IPD在50mm至75mm的范围内，45mm至80mm的较宽范围可能包括(几乎)所有成年人，而儿童(低至5岁)的最小IPD为约40mm)。在此设想，多个图像捕获模块830和可以包括一个或更多个传感器840可以被配置成图像捕获装置830与传感器840的组合，被配置成其中传感器840控制图像捕获装置830的深度或对图像捕获装置830的深度进行设置的集成单元或模块，无论图4中所示出的场景S——诸如前景、和人P或对象、背景(诸如最接近点CP、关键主体点KS和最远点FP)——中的不同深度如何。在此用于参考，多个图像捕获装置可以包括：以靠近瞳孔间距离宽度IPD的第一端IPD IPD.1的位置中心的第一数字图像捕获装置831、以靠近瞳孔间距离宽度IPD的第二端IPD.2的位置中心的第四数字图像捕获装置834、以及其余数字图像捕获装置——第二数字图像捕获装置832和第三数字图像捕获装置833，第二数字图像捕获装置832和第三数字图像捕获装置833分别在瞳孔间距离宽度IPD的第一端IPD IPD.1与第二端IPD.2之间均匀地间隔开。8A , by way of example and not limitation, a computer system 10, such as a smart device or portable smart device, is illustrated having a back side 810, a first edge such as a short edge 811, and a second edge such as a long edge 812. The back side 810 may include an I/O device 632, such as an exemplary embodiment of an image capture module 830, and may include one or more sensors 840 to measure the distance between the computer system 10 and a selected depth (depth) in an image or scene S. The image capture module 830 may include multiple or four digital image capture devices 831, 832, 833, 834, wherein the four digital image capture devices are vertically linearly positioned in series about the dorsal side 810 or close to and parallel to the long edge 812 (within the eye or interpupillary distance width IPD (the distance between the pupils of the human visual system within the comfortable circle relationship to optimize the digital multi-dimensional image of the human visual system). The interpupillary distance width IPD is preferably the distance between the pupils of an average person, and may have a distance between about two inches and two and a half inches - 2.5 inches (6.35 cm), more preferably about 40 mm to 80 mm, the IPD of most adults is in the range of 50 mm to 75 mm, the wider range of 45 mm to 80 mm may include (almost) all adults, and the minimum IPD for children (as low as 5 years old) is about 40 mm). It is contemplated herein that the plurality of image capture modules 830 and may include one or more sensors 840 may be configured as a combination of an image capture device 830 and a sensor 840, configured as an integrated unit or module in which the sensor 840 controls or sets the depth of the image capture device 830, regardless of the different depths in the scene S shown in FIG. 4, such as foreground, and a person P or object, background (such as a closest point CP, a key subject point KS, and a farthest point FP). For reference herein, the plurality of image capture devices may include: a first digital image capture device 831 centered at a position near a first end IPD IPD.1 of an interpupillary distance width IPD, a fourth digital image capture device 834 centered at a position near a second end IPD.2 of an interpupillary distance width IPD, and the remaining digital image capture devices, a second digital image capture device 832 and a third digital image capture device 833, the second digital image capture device 832 and the third digital image capture device 833 being evenly spaced between the first end IPD IPD.1 and the second end IPD.2 of the interpupillary distance width IPD, respectively.
在此设想，具有显示器的智能装置或便携式智能装置可以被配置成为矩形或方形或提供具有第一边缘811和第二边缘812的表面区域的其他类似构造。It is contemplated that a smart device or portable smart device with a display may be configured in a rectangular or square shape or other similar configurations providing a surface area having a first edge 811 and a second edge 812 .
在此设想，数字图像捕获装置831至834或图像捕获模块830可以被凹陷的、阶梯状的或斜面的边缘814包围，每个图像捕获装置831至834可以被凹陷的、阶梯状的或斜面的环816围绕，并且数字图像捕获装置831至834或图像捕获模块830可以被镜头盖820覆盖，该镜头盖下面有镜头818。It is contemplated herein that the digital image capture devices 831 to 834 or the image capture module 830 may be surrounded by a recessed, stepped or beveled edge 814, each image capture device 831 to 834 may be surrounded by a recessed, stepped or beveled ring 816, and the digital image capture devices 831 to 834 or the image capture module 830 may be covered by a lens cover 820 having a lens 818 underneath.
在此设想，数字图像捕获装置831至834可以是单独的捕获装置，而不是图像捕获模块的一部分。It is contemplated that digital image capture devices 831-834 may be separate capture devices rather than part of an image capture module.
在此进一步设想，数字图像捕获装置831至834可以设置在背侧810的任何地方，并且大体上与长边缘812平行。It is further contemplated herein that the digital image capture devices 831 - 834 may be disposed anywhere on the back side 810 and generally parallel to the long edge 812 .
在此设想，图像捕获装置可以包括定位在眼内或瞳孔间距离宽度IPD内的附加捕获装置。It is contemplated that the image capture device may include additional capture devices positioned within the eye or within the interpupillary distance width IPD.
在此进一步设想，数字图像捕获装置831至834可以用于捕获场景S的一系列2D图像。It is further contemplated herein that digital image capture devices 831 to 834 may be used to capture a series of 2D images of scene S.
现在参照图8B，通过示例而非限制的方式，示出了计算机系统10或其他智能装置或便携式智能装置，其具有背侧810、短边缘811和长边缘812。背侧810可以包括I/O装置632，诸如图像捕获模块830的示例性实施方式，并且可以包括一个或更多个传感器840，用以测量计算机系统10与图像或场景S中的选定深度(深度)之间的距离。图像捕获模块830可以包括多个或四个数字图像捕获装置831、832、833、834，其中四个数字图像捕获装置关于背侧810或靠近且平行于短边缘812(在眼内或瞳孔间距离宽度IPD(舒适圆关系内的人类视觉系统的瞳孔之间的距离，以优化人类视觉系统的数字多维图像)内竖直线性串联定位。在此设想，多个图像捕获模块830可以包括一个或更多个传感器840，可以被配置为图像捕获装置830与传感器840的组合，被配置成其中传感器840控制图像捕获装置830的深度或对图像捕获装置830的深度进行设置的集成单元或模块，诸如在场景S中的不同深度—诸如前景、背景和人P或物体—诸如最接近点CP、关键主体点KS和最远点FP，如图4中所示出的。在此用于参考，多个图像捕获装置可以包括：以靠近瞳孔间距离宽度IPD的位置中心的第一端IPD IPD.1的第一数字图像捕获装置831、以靠近瞳孔间距离宽度IPD的第二端IPD.2的位置中心的第四数字图像捕获装置834、以及其余数字图像捕获装置—第二数字图像捕获装置832和第三数字图像捕获装置833，第二数字图像捕获装置832和第三数字图像捕获装置833分别在瞳孔间距离宽度IPD的第一端IPD IPD.1与第二端IPD.2之间均匀地间隔开。8B , by way of example and not limitation, a computer system 10 or other smart device or portable smart device is shown having a back side 810, a short edge 811, and a long edge 812. The back side 810 may include an I/O device 632, such as an exemplary embodiment of an image capture module 830, and may include one or more sensors 840 to measure the distance between the computer system 10 and a selected depth (depth) in an image or scene S. The image capture module 830 may include a plurality of or four digital image capture devices 831, 832, 833, 834, wherein the four digital image capture devices are vertically linearly positioned in series about the back side 810 or close to and parallel to the short edge 812 (within the eye or the interpupillary distance width IPD (the distance between the pupils of the human visual system within the comfort circle relationship to optimize the digital multi-dimensional image of the human visual system). It is contemplated herein that the plurality of image capture modules 830 may include one or more sensors 840, which may be configured as a combination of an image capture device 830 and a sensor 840, configured as an integrated unit or module in which the sensor 840 controls the depth of the image capture device 830 or sets the depth of the image capture device 830, such as at different depths in the scene S—such as foreground, background, and person P or object—such as the closest point CP, the key subject point KS, and the farthest point FP, as shown in FIG. 4. For reference herein, the plurality of image capture devices may include: a first end IPD having a position center near the interpupillary distance width IPD; a second end IPD having a position center near the interpupillary distance width IPD; and a third end IPD having a position center near the interpupillary distance width IPD. The first digital image capture device 831 of the interpupillary distance width IPD.1, the fourth digital image capture device 834 centered at the second end IPD.2 of the interpupillary distance width IPD, and the remaining digital image capture devices - the second digital image capture device 832 and the third digital image capture device 833, the second digital image capture device 832 and the third digital image capture device 833 are respectively evenly spaced between the first end IPD IPD.1 and the second end IPD.2 of the interpupillary distance width IPD.
在此设想，数字图像捕获装置831至834或图像捕获模块830可以被凹陷的、阶梯状的或斜面的边缘814包围，每个图像捕获装置831至834可以被凹陷的、阶梯状的或斜面的环816围绕，并且数字图像捕获装置831至834或图像捕获模块830可以被镜头盖820覆盖，该镜头盖下面有镜头818。It is contemplated herein that the digital image capture devices 831 to 834 or the image capture module 830 may be surrounded by a recessed, stepped or beveled edge 814, each image capture device 831 to 834 may be surrounded by a recessed, stepped or beveled ring 816, and the digital image capture devices 831 to 834 or the image capture module 830 may be covered by a lens cover 820 having a lens 818 underneath.
在此设想，数字图像捕获装置831至834可以是单独的捕获装置，而不是图像捕获模块的一部分。It is contemplated that digital image capture devices 831-834 may be separate capture devices rather than part of an image capture module.
在此进一步设想，数字图像捕获装置831至834可以设置在背侧810的任何地方，并且大体上与长边缘812平行。It is further contemplated herein that the digital image capture devices 831 - 834 may be disposed anywhere on the back side 810 and generally parallel to the long edge 812 .
在此进一步设想，数字图像捕获装置831至834可以用于捕获场景S的一系列2D图像。It is further contemplated herein that digital image capture devices 831 to 834 may be used to capture a series of 2D images of scene S.
关于计算机系统10和图像捕获装置830，应认识到最佳尺寸关系——包括尺寸、材料、形状、形式、位置、连接、功能及操作、组装和使用方式的变化——旨在被本公开所涵盖。With respect to computer system 10 and image capture device 830, it should be recognized that optimal dimensional relationships - including variations in size, materials, shape, form, location, connection, function, and manner of operation, assembly, and use - are intended to be encompassed by the present disclosure.
在本公开内容中，瞳孔间距离宽度IPD可以具有测量宽度，以定位数字图像捕获装置831至334中心到中心在近似最大宽度115毫米到最小宽度50毫米之间；更优选地在大约72.5毫米的最大宽度至53.5毫米的最小宽度之间；以及最优选地大约64毫米的最大平均宽度至61.7毫米的最小平均宽度之间，以及图9所示人类视觉系统的63毫米(2.48英寸)中心到中心宽度的平均宽度。In the present disclosure, the interpupillary distance width IPD can have a measured width to position the digital image capture device 831 to 334 center to center between approximately a maximum width of 115 mm and a minimum width of 50 mm; more preferably between a maximum width of approximately 72.5 mm and a minimum width of 53.5 mm; and most preferably between a maximum average width of approximately 64 mm and a minimum average width of 61.7 mm, and an average width of 63 mm (2.48 inches) center to center width for the human visual system shown in FIG. 9 .
再次参照图1A、图1B、图2A、图5、图9、图14B，双眼视差是立体感知因素，它是由于左眼和右眼平均分开大约64毫米而产生的。当双眼视差相对大时，观察者会感觉到与关键主体的距离相对较近。当双眼视差相对小时，观察者会感觉到与关键主体KS的距离相对较远或较大。辐辏角V是指当眼睛聚焦在关键主体KS上时，以关键主体为顶点的左眼和右眼之间的角度。随着辐辏角增加时(随着两眼向内转动时)，观察者认为关键主体KS的距离相对较小。随着辐辏角减小时(随着双眼向外旋转时)，观察者认为关键主体KS的距离相对较大。Referring again to FIGS. 1A, 1B, 2A, 5, 9, and 14B, binocular parallax is a factor of stereoscopic perception that is caused by the left and right eyes being separated by an average of approximately 64 mm. When the binocular parallax is relatively large, the observer will feel that the distance to the key subject is relatively close. When the binocular parallax is relatively small, the observer will feel that the distance to the key subject KS is relatively far or large. The convergence angle V refers to the angle between the left eye and the right eye with the key subject as the vertex when the eyes are focused on the key subject KS. As the convergence angle increases (as the eyes turn inward), the observer perceives that the distance to the key subject KS is relatively small. As the convergence angle decreases (as the eyes rotate outward), the observer perceives that the distance to the key subject KS is relatively large.
现在参照图8C，通过示例而非限制的方式，示出了图像捕获模块830的示例性实施方式的分解图。图像捕获模块830可以包括数字图像捕获装置831至834，其中四个图像捕获装置在眼内或瞳孔间距离宽度IPD(普通人的瞳孔之间的距离)内线性地串联。数字图像捕获装置831至834可以包括第一数字图像捕获装置831、第二数字图像捕获装置832、第三数字图像捕获装置833、第四数字图像捕获装置834。第一数字图像捕获装置831可以在以靠近瞳孔间距离宽度IPD的第一端IPD.1的位置为中心，第四数字图像捕获装置834可以以靠近瞳孔间距离宽度IPD的第二端IPD.2的位置为中心，以及其余数字图像捕获装置诸如第二数字图像捕获装置832和第三数字图像捕获装置833可以定位在瞳孔间距离宽度IPD的第一端IPD.1与第二端IPD.2之间或在其之间均匀地间隔开。在一个实施方式中，每个数字图像捕获装置831至834或镜头818可以由斜边814包围、由环816围绕、和/或由镜头盖820覆盖，在镜头盖820下面有镜头818。Referring now to FIG. 8C , an exploded view of an exemplary embodiment of an image capture module 830 is shown by way of example and not limitation. The image capture module 830 may include digital image capture devices 831 to 834, wherein four image capture devices are linearly connected in series within the eye or within the interpupillary distance width IPD (the distance between the pupils of an average person). The digital image capture devices 831 to 834 may include a first digital image capture device 831, a second digital image capture device 832, a third digital image capture device 833, and a fourth digital image capture device 834. The first digital image capture device 831 may be centered at a position close to a first end IPD.1 of the interpupillary distance width IPD, the fourth digital image capture device 834 may be centered at a position close to a second end IPD.2 of the interpupillary distance width IPD, and the remaining digital image capture devices such as the second digital image capture device 832 and the third digital image capture device 833 may be positioned between the first end IPD.1 and the second end IPD.2 of the interpupillary distance width IPD or evenly spaced therebetween. In one embodiment, each digital image capture device 831 - 834 or lens 818 may be enclosed by a bevel 814 , surrounded by a ring 816 , and/or covered by a lens cover 820 , beneath which the lens 818 is located.
在此进一步设想，数字图像捕获装置831至834可以用于捕获场景S的一系列2D图像。It is further contemplated herein that digital image capture devices 831 to 834 may be used to capture a series of 2D images of scene S.
现在参照图8D，通过示例而非限制的方式，图示出了图8C的图像捕获模块830的示例性实施方式的截面图。图像捕获模块830可以包括数字图像捕获装置831至834，其中四个图像捕获装置在眼内或瞳孔间距离宽度IPD(普通人的瞳孔之间的距离)内线性地串联。数字图像捕获装置831至834可以包括第一数字图像捕获装置831、第二数字图像捕获装置832、第三数字图像捕获装置833、第四数字图像捕获装置834。每个数字图像捕获装置831至834或镜头818可以被斜边814包围、被环816围绕、和/或被镜头盖820覆盖，在镜头盖820下面有镜头818。在此设想，数字图像捕获装置831至834可以包括：光学模块，诸如镜头818，镜头818被配置成将来自场景S的光聚焦在传感器模块上，传感器模块诸如为：图像捕获传感器822图像捕获传感器822被配置城生成用于场景S的捕获图像的图像信号；以及数据处理模块824，数据处理模块824被配置为基于来自图像捕获传感器822的生成的图像信号来生成用于捕获图像的图像数据。Referring now to FIG. 8D , by way of example and not limitation, a cross-sectional view of an exemplary embodiment of the image capture module 830 of FIG. 8C is illustrated. The image capture module 830 may include digital image capture devices 831 to 834, wherein four image capture devices are linearly connected in series within an intraocular or interpupillary distance width IPD (the distance between the pupils of an average person). The digital image capture devices 831 to 834 may include a first digital image capture device 831, a second digital image capture device 832, a third digital image capture device 833, and a fourth digital image capture device 834. Each digital image capture device 831 to 834 or lens 818 may be surrounded by a bevel 814, surrounded by a ring 816, and/or covered by a lens cover 820, with a lens 818 underneath. It is envisioned herein that the digital image capture devices 831 to 834 may include: an optical module, such as a lens 818, which is configured to focus light from a scene S onto a sensor module, such as: an image capture sensor 822; the image capture sensor 822 is configured to generate an image signal for a captured image of the scene S; and a data processing module 824, which is configured to generate image data for the captured image based on the generated image signal from the image capture sensor 822.
在此设想，在此可以利用其他传感器组件822来生成用于场景S的捕获图像的图像信号，以及利用其他数据处理模块824来处理或操纵图像数据。It is contemplated that other sensor components 822 may be utilized to generate image signals for captured images of scene S, and other data processing modules 824 may be utilized to process or manipulate the image data.
在此设想，当不利用传感器840来计算场景S中的不同深度(从数字图像捕获装置831至834到前景、背景和人P或物体的距离，诸如最近的点CP、关键主体点KS和最远的点FP，如图4所示出的)，然后可以提示用户从数字图像捕获装置831至834到场景S中的关键主体点KS的设定距离来捕获场景S图像，包括但不限于从场景S的最近点CP或关键主体KS点的6英尺(6ft)距离。It is contemplated herein that when the sensor 840 is not utilized to calculate different depths in the scene S (the distances from the digital image capture devices 831 to 834 to the foreground, background, and person P or objects, such as the nearest point CP, the key subject point KS, and the farthest point FP, as shown in FIG. 4 ), the user may then be prompted to capture an image of the scene S at a set distance from the digital image capture devices 831 to 834 to the key subject point KS in the scene S, including but not limited to a distance of 6 feet (6 ft) from the nearest point CP or the key subject KS point of the scene S.
在此进一步设想，数字图像捕获装置831至834可以用于捕获场景S的一系列2D图像。It is further contemplated herein that digital image capture devices 831 to 834 may be used to capture a series of 2D images of scene S.
现在参照图8E，通过示例而非限制的方式，示出了图像捕获模块830的示例性实施方式的分解图。图像捕获模块830可以包括数字图像捕获装置831至833，其中多个或三个图像捕获装置在眼内或瞳孔间距离宽度IPD(普通人的瞳孔之间的距离)内线性地串联。数字图像捕获装置831至833可以包括第一数字图像捕获装置831、第二数字图像捕获装置832、和第三数字图像捕获装置833。第一数字图像捕获装置831可以以靠近瞳孔间距离宽度IPD的第一端IPD.1的位置为中心，第三数字图像捕获装置833可以以靠近瞳孔间距离宽度IPD的第二端IPD.2的位置为中心，其余图像捕获装置诸如第二数字图像捕获装置832可以以瞳孔间距离宽度IPDE的第一端IPD.1与第二端IPD.2之间的中心线CL为中心。在一个实施方式中，每个数字图像捕获装置831至834或镜头818可以由斜边814包围、由环816围绕、和/或由镜头盖820覆盖，在镜头盖820下面有镜头818。Referring now to FIG. 8E , an exploded view of an exemplary embodiment of an image capture module 830 is shown by way of example and not limitation. The image capture module 830 may include digital image capture devices 831 to 833, wherein a plurality or three image capture devices are linearly connected in series within the eye or within the interpupillary distance width IPD (the distance between the pupils of an average person). The digital image capture devices 831 to 833 may include a first digital image capture device 831, a second digital image capture device 832, and a third digital image capture device 833. The first digital image capture device 831 may be centered at a position close to a first end IPD.1 of the interpupillary distance width IPD, the third digital image capture device 833 may be centered at a position close to a second end IPD.2 of the interpupillary distance width IPD, and the remaining image capture devices such as the second digital image capture device 832 may be centered at a center line CL between the first end IPD.1 and the second end IPD.2 of the interpupillary distance width IPDE. In one embodiment, each digital image capture device 831 - 834 or lens 818 may be enclosed by a bevel 814 , surrounded by a ring 816 , and/or covered by a lens cover 820 , beneath which the lens 818 is located.
在此进一步设想，数字图像捕获装置831至833可以用于捕获场景S的一系列2D图像。It is further contemplated herein that the digital image capture devices 831 to 833 may be used to capture a series of 2D images of the scene S.
现在参照图8F，通过示例而非限制的方式，图示出了图8E的图像捕获模块830的示例性实施方式的截面图。图像捕获模块830可以包括数字图像捕获装置831至833，其中三个图像捕获装置在眼内或瞳孔间距离宽度IPD(普通人的瞳孔之间的距离)内线性地串联。数字图像捕获装置831至833可以包括第一数字图像捕获装置831、第二数字图像捕获装置832、和第三数字图像捕获装置833。每个数字图像捕获装置831至833或镜头818可以被斜边814包围、被环816围绕、和/或被镜头盖820覆盖，在镜头盖820下面有镜头818。在此设想，数字图像捕获装置831至833可以包括：光学模块，诸如镜头818，镜头818被配置为将场景S的图像聚焦在传感器模块上，传感器模块诸如为：图像捕获传感器822，图像捕获传感器822被配置为生成用于场景S的捕获图像的图像信号；以及数据处理模块824，数据处理模块824被配置为基于来自图像捕获传感器822的生成的图像信号来生成用于捕获图像的图像数据。Referring now to FIG. 8F , by way of example and not limitation, a cross-sectional view of an exemplary embodiment of the image capture module 830 of FIG. 8E is illustrated. The image capture module 830 may include digital image capture devices 831 to 833, wherein three image capture devices are linearly connected in series within an intraocular or interpupillary distance width IPD (the distance between the pupils of an average person). The digital image capture devices 831 to 833 may include a first digital image capture device 831, a second digital image capture device 832, and a third digital image capture device 833. Each digital image capture device 831 to 833 or lens 818 may be surrounded by a bevel 814, surrounded by a ring 816, and/or covered by a lens cover 820, with the lens 818 underneath. It is envisioned herein that the digital image capture devices 831 to 833 may include: an optical module, such as a lens 818, which is configured to focus an image of a scene S on a sensor module, such as: an image capture sensor 822, which is configured to generate an image signal for a captured image of the scene S; and a data processing module 824, which is configured to generate image data for the captured image based on the generated image signal from the image capture sensor 822.
现在参照图8G，通过示例而非限制的方式，示出了图像捕获模块830的示例性实施方式的分解图。图像捕获模块830可以包括多个或两个数字图像捕获装置831至832，其中两个图像捕获装置在眼内或瞳孔间距离宽度IPD(普通人的瞳孔之间的距离)内线性地串联。图像捕获装置831至832可以包括第一图像捕获装置831和第二图像捕获装置332。第一图像捕获装置831可以靠近瞳孔间距离宽度IPD的第一端IPD IPD.1居中，第二图像捕获装置832可以靠近瞳孔间距离宽度IPD的第二端IPD.2居中。在一个实施方式中，每个图像捕获装置831至832或镜头818可以由斜边814包围、由环816围绕、和/或由镜头盖820覆盖，在镜头盖320下面有镜头818。Referring now to FIG. 8G , an exploded view of an exemplary embodiment of an image capture module 830 is shown by way of example and not limitation. The image capture module 830 may include a plurality of or two digital image capture devices 831-832, wherein the two image capture devices are linearly connected in series within the eye or within the interpupillary distance width IPD (the distance between the pupils of an average person). The image capture devices 831-832 may include a first image capture device 831 and a second image capture device 332. The first image capture device 831 may be centered near a first end IPD IPD.1 of the interpupillary distance width IPD, and the second image capture device 832 may be centered near a second end IPD.2 of the interpupillary distance width IPD. In one embodiment, each image capture device 831-832 or lens 818 may be surrounded by a bevel 814, surrounded by a ring 816, and/or covered by a lens cover 820, with the lens 818 underneath the lens cover 320.
现在参照图8H，通过示例而非限制的方式，示出了图8G的图像捕获模块830的示例性实施方式的截面图。图像捕获模块830可以包括数字或图像捕获装置831至832，其中两个图像捕获装置在眼内或瞳孔间距离宽度IPD(普通人的瞳孔之间的距离)内线性地串联。图像捕获装置831至832可以包括第一图像捕获装置831和第二图像捕获装置332。每个图像捕获装置831至832或镜头818可以被斜边814包围、被环816围绕、和/或被镜头盖820覆盖，在镜头盖320下面有镜头818。在此设想，图像捕获装置831至832可以包括：光学模块，诸如镜头818，镜头818被配置成将场景S的图像聚焦在传感器模块上，传感器模块诸如为：图像捕获传感器822，图像捕获传感器822被配置成生成用于场景S的捕获图像的图像信号；以及数据处理模块824，数据处理模块824被配置成基于来自图像捕获传感器822的生成的图像信号来生成用于捕获图像的图像数据。Referring now to FIG. 8H , by way of example and not limitation, a cross-sectional view of an exemplary embodiment of the image capture module 830 of FIG. 8G is shown. The image capture module 830 may include digital or image capture devices 831-832, wherein two image capture devices are linearly connected in series within an intraocular or interpupillary distance width IPD (the distance between the pupils of an average person). The image capture devices 831-832 may include a first image capture device 831 and a second image capture device 332. Each image capture device 831-832 or lens 818 may be surrounded by a bevel 814, surrounded by a ring 816, and/or covered by a lens cover 820, with the lens 818 underneath. It is envisioned herein that the image capture devices 831 to 832 may include: an optical module, such as a lens 818, which is configured to focus an image of the scene S on a sensor module, such as: an image capture sensor 822, which is configured to generate an image signal for a captured image of the scene S; and a data processing module 824, which is configured to generate image data for the captured image based on the generated image signal from the image capture sensor 822.
在此设想，在此可以利用其他传感器部件来生成用于场景S的捕获图像的图像信号，以及利用其他数据处理模块824来处理或操纵图像数据。It is contemplated that other sensor components may be utilized to generate image signals for captured images of scene S, and other data processing modules 824 may be utilized to process or manipulate the image data.
在此设想，图像捕获模块830和/或数字图像捕获装置831至834用于获得场景S的偏移2D数字图像视图。此外，在此进一步设想，图像捕获模块830可以包括除本文所阐述的数量之外的多个图像捕获装置，条件是多个图像捕获装置被定位在大约眼内或瞳孔间距离宽度IPD(普通人的瞳孔间距离)内。此外，在此进一步设想，图像捕获模块830可以包括定位在大约等于瞳孔间距离宽度IPD的线性距离内的多个图像捕获装置。此外，在此进一步设想，图像捕获模块830可以包括垂直地定位的多个图像捕获装置(计算机系统10或具有短边缘811的其他智能装置或便携式智能装置)、水平地定位的多个图像捕获装置(计算机系统10或具有长边缘812的其他智能装置或便携式智能装置)或以其他方式线性地间隔开并且在近似等于瞳孔间距离宽度IPD的线性距离内的多个图像捕获装置。It is contemplated herein that the image capture module 830 and/or the digital image capture devices 831 to 834 are used to obtain offset 2D digital image views of the scene S. In addition, it is further contemplated herein that the image capture module 830 may include a plurality of image capture devices other than the number set forth herein, provided that the plurality of image capture devices are positioned within approximately the intraocular or interpupillary distance width IPD (interpupillary distance of an average person). In addition, it is further contemplated herein that the image capture module 830 may include a plurality of image capture devices positioned within a linear distance approximately equal to the interpupillary distance width IPD. In addition, it is further contemplated herein that the image capture module 830 may include a plurality of image capture devices positioned vertically (computer system 10 or other smart device or portable smart device having short edge 811), a plurality of image capture devices positioned horizontally (computer system 10 or other smart device or portable smart device having long edge 812), or a plurality of image capture devices otherwise linearly spaced apart and within a linear distance approximately equal to the interpupillary distance width IPD.
在此进一步设想，图像捕获模块830和线性地定位在眼内或瞳孔间距离宽度IPD内的数字图像捕获装置831至834使得在显示器628中能够再现准确的场景S，以在显示器628上产生多维数字图像。It is further contemplated herein that the image capture module 830 and the digital image capture devices 831 to 834 linearly positioned within the intraocular or interpupillary distance width IPD enable the accurate scene S to be reproduced in the display 628 to produce a multi-dimensional digital image on the display 628 .
现在参照图9，通过示例而非限制的方式，示出了人的面部前视图，其具有左眼LE和右眼RE且各自具有瞳孔的中点P1、P2，以说明人眼间距或眼内或瞳孔间距离IPD宽度——普通人的视觉系统瞳孔之间的距离。瞳孔间距离(IPD)是指眼睛瞳孔中心之间以毫米/英寸测量的距离。这个测量值因人而异，也取决于他们是在看近处的对象还是远处的对象。P1可以通过瞳孔间距离宽度IPD的第一端IPD.1表示，以及PS可以通过瞳孔间距离宽度IPD的第二端IPD.2表示。瞳孔间距离宽度IPD优选地是普通人的瞳孔之间的距离，可以具有大约两英寸至两英寸半——2.5英寸(6.35cm)之间、更优选地大约40mm至80mm之间的距离，绝大多数成年人的IPD在50mm至75mm的范围内，45mm至80mm的较宽范围可能包括(几乎)所有成年人，而儿童(低至5岁)的最小IPD为约40mm)。Referring now to FIG. 9 , by way of example and not limitation, a front view of a person's face is shown with a left eye LE and a right eye RE and each with a pupil midpoint P1, P2 to illustrate the interocular distance or intraocular or interpupillary distance IPD width - the distance between the pupils of the average person's visual system. The interpupillary distance (IPD) refers to the distance between the centers of the pupils of the eyes measured in millimeters/inches. This measurement varies from person to person and also depends on whether they are looking at a near object or a far object. P1 can be represented by a first end of the interpupillary distance width IPD, IPD.1, and PS can be represented by a second end of the interpupillary distance width IPD, IPD.2. The interpupillary distance width IPD is preferably the distance between the pupils of an average person and can have a distance between about two inches and two and a half inches - 2.5 inches (6.35 cm), more preferably between about 40 mm and 80 mm, with the vast majority of adults having an IPD in the range of 50 mm to 75 mm, the wider range of 45 mm to 80 mm likely including (almost) all adults, and the smallest IPD for children (as low as 5 years old) being about 40 mm).
在此设想，左图像和右图像可以如美国专利9,992,473、美国专利10,033,990和美国专利10,178,247的图6.1至图6.3所阐述那样的产生，并且以电子方式传送至左像素550L和右像素550R。此外，2D图像可以电子方式传送至中心像素550C。It is contemplated that left and right images may be generated as described in FIGS. 6.1-6.3 of US Pat. No. 9,992,473, US Pat. No. 10,033,990, and US Pat. No. 10,178,247, and electronically transmitted to left pixel 550L and right pixel 550R. Additionally, a 2D image may be electronically transmitted to center pixel 550C.
现在参照图10，通过示例而非限制的方式，图示出了按图4和图3的比例的舒适圆(CoC)的代表性图示。对于定义的平面，如果图像的很大一部分是在舒适圆(CoC)内捕获的，那么在镜头平面上捕获的图像将是舒适的，并且与观看显示在显示器628上的最终图像的用户U的人类视觉系统兼容。当由两个图像捕获装置诸如图像捕获装置831至833或图像捕获装置831至834(瞳孔间距离IPD)在舒适圆(CoC)内捕获的任何物体—诸如近平面N、关键主体平面KSP和远平面FP—再现为可在显示器628上观看的数字多维图像序列时，对观看者来说其将处于焦点。背侧物体平面或远平面FP可以定义为到15度径向线到视场中的垂直线到30度线或R的交点的距离，R为舒适圆(CoC)的半径。此外，将舒适圆(CoC)定义为通过使圆的直径沿关键主体KS平面(KSP)的垂直线通过(其中宽度由镜头平面—图像捕获模块830—上的中心点的30度径向决定)而形成的圆。Referring now to FIG. 10 , by way of example and not limitation, a representative illustration of a comfort circle (CoC) to the scale of FIGS. 4 and 3 is shown. For a defined plane, if a significant portion of the image is captured within the comfort circle (CoC), then the image captured on the lens plane will be comfortable and compatible with the human visual system of a user U viewing the final image displayed on the display 628. Any object captured within the comfort circle (CoC) by two image capture devices, such as image capture devices 831 to 833 or image capture devices 831 to 834 (interpupillary distance IPD)—such as the near plane N, the key subject plane KSP, and the far plane FP—will be in focus for the viewer when reproduced as a digital multi-dimensional image sequence viewable on the display 628. The dorsal object plane or far plane FP may be defined as the distance to the intersection of a 15 degree radial line to a vertical line in the field of view to a 30 degree line or R, where R is the radius of the comfort circle (CoC). Additionally, a comfort circle (CoC) is defined as a circle formed by passing the diameter of the circle along a perpendicular line to the key subject KS plane (KSP), where the width is determined by a 30 degree radial from the center point on the lens plane - image capture module 830.
可以利用图像捕获装置诸如数字图像捕获装置831至833或数字图像捕获装置831至834(瞳孔间距离IPD)在刚好与舒适圆(CoC)相切的30度线内的线性定位或间隔，以在观看可在显示器628上观看的数字多维图像序列时在多个图像之间产生运动视差，这将是舒适的且与用户U的人类视觉系统兼容。Image capture devices such as digital image capture devices 831 to 833 or digital image capture devices 831 to 834 (interpupillary distance IPD) can be linearly positioned or spaced within a 30 degree line just tangent to the comfort circle (CoC) to produce motion parallax between multiple images when viewing a digital multi-dimensional image sequence viewable on display 628, which will be comfortable and compatible with the human visual system of user U.
现在参照图10A、图10B、图10C和图11，通过示例而非限制的方式，示出了从图10得出的直角三角形。所有的定义都是基于在场景与图像捕获的关系内保持直角三角形。因此，知道了关键主体KS距离(会聚点)，我们可以计算出以下参数。Referring now to Figures 10A, 10B, 10C and 11, by way of example and not limitation, a right triangle derived from Figure 10 is shown. All definitions are based on maintaining a right triangle within the relationship of the scene to the image capture. Thus, knowing the key subject KS distance (convergence point), we can calculate the following parameters.
图6A用于计算舒适(CoC)的半径R。FIG6A is used to calculate the radius R of comfort (CoC).
R/KS＝tan 30°R/KS＝tan 30°
R＝KS*tan 30°R＝KS*tan 30°
图6B用于计算图像捕获装置(诸如图像捕获装置831至833或图像捕获装置831至834)之间的最佳距离(瞳孔间距离IPD)。FIG. 6B is for calculating an optimal distance (inter-pupillary distance IPD) between image capturing devices (such as image capturing devices 831 to 833 or image capturing devices 831 to 834 ).
TR/KS＝tan 15°TR/KS＝tan 15°
TR＝KS*tan 15°；并且IPD为2*TRTR = KS*tan 15°; and IPD is 2*TR
如图6C计算最佳远平面FPThe best far plane FP is calculated as shown in Figure 6C
Tan 15°＝R/BTan 15°＝R/B
B＝(KS*tan 30°)/tan 15°B＝(KS*tan 30°)/tan 15°
近平面NP与远平面FP之比＝((KS/(KS 8tan 30°))*tan 15°Ratio of near plane NP to far plane FP = ((KS/(KS 8tan 30°))*tan 15°
为了理解TR的含义，在镜头平面的线性图像捕获线上的点，15度线击中/触及舒适(CoC)。将图像布置成使得关键主体KS点在所有通过图像捕获装置诸如数字图像捕获装置831至833或数字图像捕获装置831至834捕获的图像中是相同的。To understand what TR means, at a point on the linear image capture line of the lens plane, the 15 degree line hits/touches comfort (CoC). The images are arranged so that the key subject KS point is the same in all images captured by an image capture device such as digital image capture device 831 to 833 or digital image capture device 831 to 834.
图像捕获装置诸如数字图像捕获装置831至833或数字图像捕获装置831至834的用户组成场景S，并且在我们的案例中移动数字图像捕获装置830，使得弥散圆传达场景S。由于数字图像捕获装置830使用线性间隔开的多相机，因此在数字图像捕获装置830诸如数字图像捕获装置831至833或数字图像捕获装置831至834的线性偏移所捕获的多个图像或帧之间存在双眼视差。这种视差可以通过以下来改变：改变数字图像捕获装置830的设置或将关键主体KS向后或远离数字图像捕获装置以减少视差，或将关键主体KS向数字图像捕获装置靠近以增加视差。我们的系统是固定的数字图像捕获装置系统，作为实验开展的准则，近平面NP应该距离数字图像捕获装置830不小于大约6英尺。The user of the image capture device, such as the digital image capture devices 831 to 833 or the digital image capture devices 831 to 834, composes a scene S, and moves the digital image capture device 830 in our case so that the circle of confusion conveys the scene S. Since the digital image capture device 830 uses multiple cameras spaced apart linearly, there is binocular parallax between multiple images or frames captured by the linear offset of the digital image capture device 830, such as the digital image capture devices 831 to 833 or the digital image capture devices 831 to 834. This parallax can be changed by changing the settings of the digital image capture device 830 or moving the key subject KS backward or away from the digital image capture device to reduce the parallax, or moving the key subject KS closer to the digital image capture device to increase the parallax. Our system is a fixed digital image capture device system, and as a guideline for the experiment to be conducted, the near plane NP should be no less than about 6 feet away from the digital image capture device 830.
现在参照图12，图示出了由计算机系统10执行的且可在显示器628上查看的方法的流程步骤1200，所述方法具有：捕获场景S的多个2D图像、生成帧1101至1104、操作、重新配置、处理、显示、存储数字多维图像序列。注意在图12中，指定手动操作模式的一些步骤可以由用户U执行，其中用户在步骤中进行选择并向计算机系统10提供输入，而否则计算机系统10的操作是基于由应用程序624在自动模式下执行的步骤的。Referring now to FIG. 12 , there is shown a flow chart of steps 1200 of a method performed by the computer system 10 and viewable on the display 628 , the method having: capturing multiple 2D images of a scene S, generating frames 1101 to 1104, manipulating, reconfiguring, processing, displaying, storing a sequence of digital multi-dimensional images. Note that in FIG. 12 , some steps specifying a manual operation mode may be performed by a user U, wherein the user makes selections in the steps and provides input to the computer system 10 , while otherwise the operation of the computer system 10 is based on steps performed by the application 624 in an automatic mode.
在块或步骤1210中，提供计算机系统10，所述计算机系统10如以上图1至图11所述具有数字图像捕获装置830、显示器628和应用624，以使捕获多个二维(2D)图像并且所述计算机系统10在显示器628上显示三维(3D)图像序列，所述多个二维(2D)图像由于数字图像捕获装置831至833、数字图像捕获装置831至834等的间距在大约眼内或瞳孔间距离宽度IPD(普通人的瞳孔之间的距离)内而具有视差。此外，在显示器628上顺序显示数字图像(DIFY或立体3D)，其中由捕获装置831至834(n个装置)捕获的场景S的多个2D图像中的图像(n)以序列顺序显示在显示器628上作为数字多维图像序列(DIFY或立体3D)。In block or step 1210, a computer system 10 is provided, the computer system 10 having a digital image capture device 830, a display 628, and an application 624 as described above in FIGS. 1 to 11, so that a plurality of two-dimensional (2D) images are captured and the computer system 10 displays a three-dimensional (3D) image sequence on the display 628, the plurality of two-dimensional (2D) images having parallax due to the spacing of the digital image capture devices 831 to 833, the digital image capture devices 831 to 834, etc. within about the intraocular or interpupillary distance width IPD (the distance between the pupils of an average person). In addition, digital images (DIFY or stereoscopic 3D) are sequentially displayed on the display 628, wherein images (n) of the plurality of 2D images of a scene S captured by the capture devices 831 to 834 (n devices) are displayed in a sequential order on the display 628 as a sequence of digital multi-dimensional images (DIFY or stereoscopic 3D).
在块或步骤1215中，计算机系统10通过图像捕获应用624(捕获方法)被配置成通过图像捕获模块830捕获多个2D数字源图像来捕获场景S的多个数字图像，该图像捕获模块830具有多个图像捕获装置，诸如线性地定位在眼内或瞳孔间距离宽度IPD(在舒适圆关系内用于优化人类视觉系统的数字多维图像的人类视觉系统的瞳孔之间的距离)内的数字图像捕获装置831-834、830(n个装置)等。计算机系统10集成I/O装置632与计算机系统10，I/O装置632可以包括与计算机系统10通信的一个或更多个传感器840，以测量计算机系统10(图像捕获装置，诸如数字图像捕获装置831-834、830(n个装置)与场景S中的选定深度(深度)诸如关键主体KS之间的距离，并且对一个或更多个数字图像捕获装置831-834、830(n个装置)的焦点进行设置。In block or step 1215, the computer system 10 is configured to capture multiple digital images of the scene S by capturing multiple 2D digital source images through an image capture module 830 having multiple image capture devices, such as digital image capture devices 831-834, 830 (n devices) linearly positioned within the eye or inter-pupillary distance width IPD (the distance between pupils of the human visual system within the comfort circle relationship for optimizing the digital multi-dimensional image of the human visual system) and the like through the image capture application 624 (capture method). The computer system 10 integrates an I/O device 632 with the computer system 10, and the I/O device 632 may include one or more sensors 840 in communication with the computer system 10 to measure the distance between the computer system 10 (image capture device, such as the digital image capture device 831-834, 830 (n devices) and a selected depth (depth) such as a key subject KS in the scene S, and to set the focus of the one or more digital image capture devices 831-834, 830 (n devices).
3D立体，用户U可以点击或与选择框812进行其他识别交互以选择或标识场景S的源图像对、左图像1102和右图像1103中的关键主体KS，如图16所示。此外，在块或步骤1215中，利用计算机系统10、显示器628和应用程序206(通过图像捕获应用)的设置来对准或在显示器628上显示的场景S的关键主体KS上定位图标，诸如图16B的十字线814，例如通过触摸或拖动场景S的图像，或将计算机系统10指向不同方向以在场景S的关键主体KS上对准图16的十字线814。在块或步骤1215中，获得或捕获聚焦于场景S的图像或场景(深度)中的选定深度的场景S)的图像(n)。3D stereo, the user U can click or perform other identification interactions with the selection box 812 to select or identify the key subject KS in the source image pair, left image 1102 and right image 1103 of the scene S, as shown in Figure 16. In addition, in block or step 1215, the settings of the computer system 10, the display 628 and the application 206 (through the image capture application) are used to align or position an icon, such as the crosshairs 814 of Figure 16B, on the key subject KS of the scene S displayed on the display 628, for example, by touching or dragging the image of the scene S, or pointing the computer system 10 in different directions to align the crosshairs 814 of Figure 16 on the key subject KS of the scene S. In block or step 1215, an image (n) of the scene S) focused on the image of the scene S or the selected depth in the scene (depth) is obtained or captured.
替代地，计算机系统10通过图像操纵应用624和显示器628可以被配置成在自动模式下操作，其中一个或更多个传感器840可以对计算机系统10(图像捕获装置，诸如数字图像捕获装置831-834、830(n个装置)与场景S中的选定深度(深度)诸如关键主体KS之间的距离进行测量。替代性地，在手动模式下，用户可以确定计算机系统10与场景S(深度)中的选定深度诸如关键主体KS之间的正确距离。Alternatively, the computer system 10, via the image manipulation application 624 and the display 628, may be configured to operate in an automatic mode, wherein one or more sensors 840 may measure the distance between the computer system 10 (image capture device, such as the digital image capture devices 831-834, 830 (n devices) and a selected depth (depth) in the scene S, such as a key subject KS. Alternatively, in a manual mode, a user may determine the correct distance between the computer system 10 and a selected depth in the scene S (depth), such as a key subject KS.
在此认识到，可以指示用户U：通过计算机系统10通过图像捕获应用624和显示器628来捕获场景S的图像(n)的最佳做法，例如对场景S进行取景以包括场景S中的关键主体KS，选择场景S的突出前景特征，以及场景S中的最远点FP，可以包括识别场景S中的关键主体KS、选择场景S中的最近点CP、场景S的突出背景特征等。此外，将场景S中的关键主体KS定位为距离与数字图像捕获装置831至834(n个装置)特定距离。此外，将场景S中最接近的点CP定位为距离数字图像捕获装置831至834(n个装置)特定距离。It is recognized here that the user U may be instructed on best practices for capturing an image (n) of a scene S through the computer system 10 through the image capture application 624 and the display 628, such as framing the scene S to include a key subject KS in the scene S, selecting a prominent foreground feature of the scene S, and a farthest point FP in the scene S, which may include identifying the key subject KS in the scene S, selecting a closest point CP in the scene S, a prominent background feature of the scene S, etc. In addition, the key subject KS in the scene S is positioned at a specific distance from the digital image capture devices 831 to 834 (n devices). In addition, the closest point CP in the scene S is positioned at a specific distance from the digital image capture devices 831 to 834 (n devices).
现在参照图13，通过示例而非限制的方式，图示出了触摸屏显示器628，使用户U能够选择计算机系统10的摄影选项。第一示例性选项可以是DIFY捕获，其中用户U可以指定或选择数字图像速度设置1302，其中用户U可以增加或减少由捕获装置831至834(n个装置)捕获的数字图像在显示器628上的顺序显示的回放速度或帧(图像)/秒。此外，用户U可以指定或选择数字图像的循环数或重复数1304，以设置由捕获装置831至834(n个装置)捕获的场景S的多个2D图像的图像(n)1000的循环数，其中由捕获装置831至834(n个装置)、830(n个装置)捕获的场景S的多个2D图像中的图像(n)1000以顺序显示在显示器628上，与图11类似。仍然进一步地，用户U可以指定或选择用于回放的数字图像序列或回文序列的回放顺序1306，以设置由捕获装置备831至834(n个装置)、830(n个装置)捕获的场景S的多个2D图像中的图像(n)1000的显示顺序。图像的定时顺序显示通过运动追逐比效应产生适当的双眼视差。在此设想计算机系统10和应用程序624可以利用在此的默认或自动设置。Referring now to FIG. 13 , by way of example and not limitation, a touch screen display 628 is illustrated that enables a user U to select photography options for the computer system 10. A first exemplary option may be DIFY capture, where the user U may specify or select a digital image speed setting 1302, where the user U may increase or decrease the playback speed or frames (images)/second of the sequential display of digital images captured by the capture devices 831 to 834 (n devices) on the display 628. In addition, the user U may specify or select a loop number or repetition number 1304 of the digital image to set the loop number of images (n) 1000 of the plurality of 2D images of the scene S captured by the capture devices 831 to 834 (n devices), 830 (n devices), where the images (n) 1000 of the plurality of 2D images of the scene S captured by the capture devices 831 to 834 (n devices), 830 (n devices) are sequentially displayed on the display 628, similar to FIG. 11 . Still further, the user U may specify or select a playback order 1306 of a digital image sequence or palindrome sequence for playback to set the display order of images (n) 1000 in a plurality of 2D images of a scene S captured by the capture devices 831 to 834 (n devices), 830 (n devices). The timed sequential display of the images produces appropriate binocular disparity through a motion pursuit ratio effect. It is contemplated that the computer system 10 and the application 624 may utilize default or automatic settings herein.
现在参照图16B，通过示例而非限制的方式，图示出了触摸屏显示器628，使用户U能够选择计算机系统10的摄影选项(3D立体声)。在块或步骤1215中，利用计算机系统10、显示器628和应用程序624(通过图像捕获应用)的设置来在显示器628上显示的场景S的关键主体KS上对准或定位图标，诸如十字线1310，例如通过触摸或拖动场景S的图像，或将计算机系统10指向不同方向以在场景S的关键主体KS上对准十字线1310。在块或步骤1215中，从图像捕获装置831至834(n个装置)获得或捕获聚焦于场景S的图像或场景(深度)中的选定深度的场景S的图像(n个)。用户U可以点击选择框1312或与其进行其他识别交互，以选择或者识别源图像对，从图像捕获装置831至834(n个装置)的场景S的图像(n个)1101、1102、1103、1104(帧集1100)中选择的场景S的左图像1102L和右图像1103R，或者图像(n个)1101、1102、1103、1104(帧集1100)中的两个图像的任意组合的中的关键主体KS。而且，计算机系统10经由图像操纵应用和显示器624可以被配置成使得用户U能够选择或识别场景S的图像作为场景S的左图像1102和右图像1103。用户U可以点击或与选择框进行其他识别交互812以选择或识别场景S的源图像对、左图像1102和右图像1103中的关键主体KS，如图16B所示。Referring now to FIG. 16B , by way of example and not limitation, a touch screen display 628 is illustrated that enables a user U to select a photography option (3D stereo) of the computer system 10. In block or step 1215, the settings of the computer system 10, the display 628, and the application 624 (via the image capture application) are utilized to align or position an icon, such as a crosshair 1310, on a key subject KS of the scene S displayed on the display 628, for example by touching or dragging the image of the scene S, or pointing the computer system 10 in different directions to align the crosshair 1310 on the key subject KS of the scene S. In block or step 1215, an image (n) of the scene S focused on or at a selected depth in the scene (depth) is obtained or captured from the image capture devices 831 to 834 (n devices). The user U may click on the selection box 1312 or perform other identification interactions therewith to select or identify the key subject KS in the source image pair, the left image 1102L and the right image 1103R of the scene S selected from the images (n) 1101, 1102, 1103, 1104 (frame set 1100) of the scene S of the image capture devices 831 to 834 (n devices), or any combination of two images in the images (n) 1101, 1102, 1103, 1104 (frame set 1100). Moreover, the computer system 10, via the image manipulation application and the display 624, may be configured to enable the user U to select or identify the image of the scene S as the left image 1102 and the right image 1103 of the scene S. The user U may click on or perform other identification interactions 812 with the selection box to select or identify the key subject KS in the source image pair, the left image 1102 and the right image 1103 of the scene S, as shown in FIG16B.
替代地，在块或步骤1215中，用户U可以利用计算机系统10、显示器628和应用程序624诸如通过AirDrop、DROP OX或其他应用来输入场景S的多个图像、文件和数据集(Dataset)。Alternatively, in block or step 1215, the user U may utilize the computer system 10, display 628, and application 624 to input multiple images, files, and datasets of the scene S, such as through AirDrop, DROP OX, or other applications.
在此认识到，步骤1215，计算机系统10通过图像捕获应用624、图像操纵应用624、图像显示应用624可以利用不同的和单独定位的计算机系统10，诸如一个或更多个用户系统720、第一智能装置722、第二智能装置724、智能装置(多个智能装置)和应用程序624来执行。例如，使用远离图像操纵系统的且远离图像浏览系统的相机系统，步骤1215可以通过计算机系统10(第一处理器)和在用户系统720、722、724与应用程序624之间通信的应用程序624来执行接近场景S。在此，相机系统可以被定位或固定以捕获事件或娱乐的不同视点的片段，例如场景S。接下来，通过通信链路740和/或网络750或5G，计算机系统10和应用程序624通过更多个用户系统720、722、724可以从捕获装置831至834(n个装置)相对于关键主体KS点捕获和传输场景S的多个数字图像，作为场景S的数字多维图像序列(DIFY)、场景S图像(n)集。It is recognized herein that step 1215, the computer system 10 via the image capture application 624, the image manipulation application 624, the image display application 624 may be performed utilizing different and separately located computer systems 10, such as one or more of the user systems 720, the first smart device 722, the second smart device 724, the smart device(s), and the application 624. For example, using a camera system remote from the image manipulation system and remote from the image viewing system, step 1215 may be performed by the computer system 10 (first processor) and the application 624 communicating between the user systems 720, 722, 724 and the application 624 to approach the scene S. Here, the camera system may be positioned or fixed to capture segments of different viewpoints of an event or entertainment, such as the scene S. Next, through the communication link 740 and/or the network 750 or 5G, the computer system 10 and the application 624 can capture and transmit multiple digital images of the scene S from the capture devices 831 to 834 (n devices) relative to the key subject KS point through more user systems 720, 722, 724 as a digital multidimensional image sequence (DIFY) of the scene S, a set of scene S images (n).
在瞳孔间距离IPD处或附近捕获的图像与人类视觉系统相匹配，这简化了数学运算，最大限度地减少了两个图像之间的串扰，减少了模糊性和图像移动，以产生可在显示器628上查看的数字多维图像序列(DIFY)。Images captured at or near the interpupillary distance IPD are matched to the human visual system, which simplifies the mathematical operations, minimizes crosstalk between the two images, and reduces blurriness and image movement to produce a digital multi-dimensional image sequence (DIFY) that can be viewed on display 628.
此外，在块或步骤1215中，利用计算机系统10、显示器628和应用程序624(通过图像捕获应用)的设置来对准或在显示器628上显示的场景S的关键主体KS上定位图标，诸如图13或图16B的十字线1310，例如通过触摸或拖动场景S的数据集，或触摸和拖动关键主体KS，或将计算机系统10指向不同方向以在场景S的关键主体KS上对准图13或图16B的十字线1310。在块或步骤1215中，从多个捕获装置830(n个装置)获得或捕获聚焦于场景S的图像或场景(深度)中的选定深度上的场景S的数据集(Dataset)。In addition, in block or step 1215, the settings of the computer system 10, the display 628, and the application 624 (via the image capture application) are used to align or position an icon, such as the crosshairs 1310 of FIG. 13 or 16B, on the key subject KS of the scene S displayed on the display 628, for example, by touching or dragging the dataset of the scene S, or touching and dragging the key subject KS, or pointing the computer system 10 in different directions to align the crosshairs 1310 of FIG. 13 or 16B on the key subject KS of the scene S. In block or step 1215, a dataset (Dataset) of the scene S focused on an image of the scene S or at a selected depth in the scene (depth) is obtained or captured from multiple capture devices 830 (n devices).
此外，在块或步骤1215中，将I/O装置632与计算机系统10集成，I/O装置632可以包括与计算机系统10通信的一个或更多个传感器852，以对计算机系统10/捕获装置830(n个装置)与场景S(深度)中的选定深度诸如关键主体KS之间的距离进行测量，并且对运载工具400和捕获装置830的弧线或轨迹的焦点进行设置。在此设想，计算机系统10、显示器628和应用程序624可以在自动模式下操作，其中一个或更多个传感器840可以对场景S(深度)中的选定深度诸如关键主体KS之间的距离进行测量。替代性地，在手动模式下，用户可以确定用户U与场景S(深度)中的选定深度诸如关键主体KS之间的正确距离。或者计算机系统10、显示器628可以利用一个或更多个传感器852来对捕获装置830(n个装置)与场景S中的选定深度(深度)诸如关键主体KS之间的距离进行测量，并且在屏幕上提供指示或消息(距离优选度)，以指示用户U将捕获装置830(n个装置)移动得离关键主体KS或近平面NP更近或更远，以优化捕获装置830(n个装置)和场景S的图像、文件和数据集(Dataset)。Additionally, in block or step 1215, an I/O device 632 is integrated with the computer system 10, and the I/O device 632 may include one or more sensors 852 that communicate with the computer system 10 to measure the distance between the computer system 10/capture device 830 (n devices) and a selected depth in the scene S(depth), such as a key subject KS, and to set the focus of the arc or trajectory of the vehicle 400 and the capture device 830. It is contemplated that the computer system 10, display 628, and application 624 may operate in an automatic mode, wherein one or more sensors 840 may measure the distance between a selected depth in the scene S(depth), such as a key subject KS. Alternatively, in a manual mode, the user may determine the correct distance between the user U and a selected depth in the scene S(depth), such as a key subject KS. Alternatively, the computer system 10, the display 628 may utilize one or more sensors 852 to measure the distance between the capture device 830 (n devices) and a selected depth (depth) in the scene S, such as a key subject KS, and provide an indication or message (distance preference) on the screen to instruct the user U to move the capture device 830 (n devices) closer to or farther away from the key subject KS or the near plane NP to optimize the images, files, and datasets (Dataset) of the capture device 830 (n devices) and the scene S.
在块或步骤1220中，计算机系统10通过图像操纵应用624被配置为通过图像获取应用接收由数字图像捕获装置831至834(n个装置)、830(n个装置)捕获的场景S的多个图像。该图像获取应用将每个图像转换为数字源图像，诸如JPEG、GIF、TIF格式。理想情况下，每个数字源图像在其中包括一些可见物体、主体或点，如与近平面NP、远平面FP相关的前景或最近的点或与远平面FP相关的最远的点、以及关键主体KS。近平面NP、远平面FP点分别是离观看者(多个捕获装置831和832、833或834、830(n个装置))最近的点和最远的点。景深是在物场内形成的深度或距离(描绘的前景到背景之间的距离)。主轴是垂直于场景通过关键主体KS点的线，而视差是关键主体KS点从主轴的位移，参见图11。在数字合成中，移位总是保持为与主轴的整数个像素。In block or step 1220, the computer system 10 is configured through the image manipulation application 624 to receive multiple images of a scene S captured by digital image capture devices 831 to 834 (n devices), 830 (n devices) through an image acquisition application. The image acquisition application converts each image into a digital source image, such as a JPEG, GIF, TIF format. Ideally, each digital source image includes some visible objects, subjects or points therein, such as the foreground or the nearest point associated with the near plane NP, the far plane FP or the farthest point associated with the far plane FP, and the key subject KS. The near plane NP and the far plane FP points are the nearest point and the farthest point from the viewer (multiple capture devices 831 and 832, 833 or 834, 830 (n devices)), respectively. Depth of field is the depth or distance formed within the object field (the distance between the depicted foreground and the background). The principal axis is a line perpendicular to the scene through the key subject KS point, and the parallax is the displacement of the key subject KS point from the principal axis, see Figure 11. In digital synthesis, the shift is always kept to an integer number of pixels from the major axis.
在此认识到，步骤1220，计算机系统10通过图像捕获应用624、图像操纵应用624、图像显示应用624可以利用不同的和单独的计算机系统10诸如一个或更多个用户系统720、722、724和应用程序624来执行。例如，使用远离图像捕获系统且远离图像查看系统的图像操纵系统，步骤1220可以通过计算机系统10(第三处理器)和在用户系统720、222、224和应用程序624之间通信的应用程序远离场景S执行。接下来，通过通信链路740和/或网络750、或5G计算机系统10(第三处理器)和应用程序624通过更多个用户系统720、722、724可以从捕获装置831至834(n个装置)相对于关键主体KS点接收场景S的多个图像(n)的集合，并且将经操纵后的场景的多个数字多维图像序列(DIFY或3D立体)传输到计算机系统10(第一处理器)和应用程序624(步骤1220A)。It is recognized herein that step 1220, the computer system 10 via the image capture application 624, the image manipulation application 624, the image display application 624 may be performed using different and separate computer systems 10 such as one or more of the user systems 720, 722, 724 and the application programs 624. For example, using an image manipulation system that is remote from the image capture system and remote from the image viewing system, step 1220 may be performed remote from the scene S via the computer system 10 (third processor) and the application programs that communicate between the user systems 720, 222, 224 and the application programs 624. Next, a set of multiple images (n) of the scene S can be received from the capture devices 831 to 834 (n devices) relative to the key subject KS point through more user systems 720, 722, 724 via the communication link 740 and/or the network 750, or the 5G computer system 10 (third processor) and the application 624, and multiple digital multi-dimensional image sequences (DIFY or 3D stereo) of the manipulated scene can be transmitted to the computer system 10 (first processor) and the application 624 (step 1220A).
在块或步骤1220A中，计算机系统10通过自动关键主体选择算法或关键主体应用程序624被配置为在由数字图像捕获装置831至834(n个装置)捕获的场景S的每个源图像、多个图像中识别关键主体KS。此外，计算机系统10通过关键主体、应用程序624被配置为至少部分地识别来自数字图像捕获装置831至834(n个装置)的场景S的一个或更多个多个图像(n)中的像素、像素组(显示器628上的手指点选择)分别作为关键主体KS。此外，计算机系统10通过关键主体应用程序624被配置为将源图像、由数字图像捕获装置831至834(n个装置)捕获的场景S的多个图像围绕关键主体KS以一定距离水平对准；(水平图像平移(HIT)，如图11A和图11B中所示，关键主体KS在舒适圆关系内，以便优化人类视觉系统的数字多维图像/序列1010。In block or step 1220A, the computer system 10 is configured via an automatic key subject selection algorithm or key subject application 624 to identify a key subject KS in each source image, multiple images of a scene S captured by digital image capture devices 831 to 834 (n devices). In addition, the computer system 10 is configured via the key subject, application 624 to at least partially identify pixels, groups of pixels (selected by finger points on display 628) in one or more multiple images (n) of the scene S from the digital image capture devices 831 to 834 (n devices), respectively, as key subjects KS. In addition, the computer system 10 is configured via the key subject application 624 to horizontally align the source images, multiple images of the scene S captured by the digital image capture devices 831 to 834 (n devices) around the key subject KS at a certain distance; (Horizontal Image Translation (HIT), as shown in Figures 11A and 11B, the key subject KS is within the comfort circle relationship so as to optimize the digital multi-dimensional image/sequence 1010 for the human visual system.
此外，识别场景S的一系列2D图像中的关键主体点，并且将场景的一系列2D图像中的每一个都与关键主体KS点对准，并且将场景的一系列2D图像中的所有其他点基于多个数字图像捕获装置的间距移位，以生成经修改的2D图像序列或经修改的2D图像对。In addition, key subject points in a series of 2D images of the scene S are identified, and each of the series of 2D images of the scene is aligned with the key subject KS point, and all other points in the series of 2D images of the scene are shifted based on the spacing of multiple digital image capture devices to generate a modified 2D image sequence or a modified 2D image pair.
如图11A、图11B和图4所示，在各个由数字图像捕获装置831至834(n个装置)捕获的场景S的多个图像中，可以识别对应于场景S的同一关键主体KS的关键主体KS。在此设想，计算机系统10、显示器628和应用程序624可以执行算法或一组步骤来自动识别由数字图像捕获装置831至834(n个装置)捕获的场景S的多个图像中的主体KS。替代地，在块或步骤1220A中，利用计算机系统10(在手动模式下——手动关键主体选择算法)、显示器628和应用程序624的设置，以至少部分地使用户U对准或编辑对准由数字图像捕获装置831至834(n个装置)捕获的场景S的多个图像的至少两个图像(n)的像素、像素组(手指点选择)、关键主体KS点。As shown in FIG. 11A , FIG. 11B and FIG. 4 , in a plurality of images of a scene S captured by each of the digital image capture devices 831 to 834 (n devices), a key subject KS corresponding to the same key subject KS of the scene S can be identified. It is contemplated that the computer system 10, the display 628 and the application 624 can execute an algorithm or a set of steps to automatically identify the subject KS in the plurality of images of the scene S captured by the digital image capture devices 831 to 834 (n devices). Alternatively, in block or step 1220A, the settings of the computer system 10 (in manual mode—manual key subject selection algorithm), the display 628 and the application 624 are utilized to at least partially enable the user U to align or edit the pixels, pixel groups (finger point selection), key subject KS points of at least two images (n) of the plurality of images of the scene S captured by the digital image capture devices 831 to 834 (n devices).
场景S的源图像、由数字图像捕获装置831至834(n个装置)捕获的场景S的多个图像都是用数字图像捕获装置831至834(n个装置)以相同的图像捕获距离和相同的焦距获得。计算机系统10通过关键主体应用624通过对源图像、由数字图像捕获装置831至834(n个装置)捕获的场景S的多个图像进行水平图像移位，从而创建一个确定点，即关键主体KS点，如图13所示，源图像、由数字图像捕获装置831至834(n个装置)捕获的场景S的多个图像在这一个点重叠。这种图像移位做了两件事，首先它设定了图像的深度。关键主体KS点前面的所有点都更接近观察者，关键主体KS点后面的所有点都距离观察者更远。The source image of scene S, the multiple images of scene S captured by digital image capture devices 831 to 834 (n devices), are all obtained by digital image capture devices 831 to 834 (n devices) at the same image capture distance and the same focal length. The computer system 10, through the key subject application 624, horizontally shifts the source image, the multiple images of scene S captured by digital image capture devices 831 to 834 (n devices), thereby creating a certain point, namely the key subject KS point, as shown in FIG. 13, where the source image, the multiple images of scene S captured by digital image capture devices 831 to 834 (n devices) overlap. This image shift does two things. First, it sets the depth of the image. All points in front of the key subject KS point are closer to the observer, and all points behind the key subject KS point are farther from the observer.
此外，在自动模式下，计算机系统10通过图像操纵应用可以基于源图像、由数字图像捕获装置831至834(n个装置)捕获的场景S的多个图像的深度图来识别关键主体KS。Furthermore, in the automatic mode, the computer system 10 through the image manipulation application may identify the key subject KS based on the source image, depth maps of multiple images of the scene S captured by the digital image capture devices 831 to 834 (n devices).
计算机系统10通过图像操纵应用可以使用源图像、由数字图像捕获装置831至834(n个装置)捕获的场景S的多个图像的深度图来识别前景、最近的点和背景、最远的点。替代地，在手动模式下，计算机系统10通过图像操纵应用和显示器628可以被配置为使用户U能够在场景S的源图像、由数字图像捕获装置831至834(n个装置)捕获的场景S的多个图像中选择或识别关键主体KS，用户U可以点击、移动光标或方框或其他识别来选择或识别场景S的源图像、由数字图像捕获装置831至834(n个装置)捕获的场景S的多个图像中的关键主体KS，如图13所示。The computer system 10, through the image manipulation application, can use the source image, the depth map of multiple images of the scene S captured by the digital image capture devices 831 to 834 (n devices) to identify the foreground, the nearest point and the background, the farthest point. Alternatively, in the manual mode, the computer system 10, through the image manipulation application and the display 628, can be configured to enable the user U to select or identify the key subject KS in the source image of the scene S, the multiple images of the scene S captured by the digital image capture devices 831 to 834 (n devices), and the user U can click, move the cursor or box or other identification to select or identify the key subject KS in the source image of the scene S, the multiple images of the scene S captured by the digital image capture devices 831 to 834 (n devices), as shown in FIG. 13.
水平图像平移(HIT)将关键主体平面KSP设定为产生场景的屏幕平面(第一平面或近侧平面)。这个步骤也设定了物体的运动，诸如近平面NP(第三平面或近平面)的灌木B和远平面FP(第二平面或远平面)的树T相对于彼此的运动。关键主体KS或关键主体平面KSP前面的物体向一个方向移动(从左到右或从右到左)，而关键主体KS或关键主体平面KSP后面的物体则以与前面的物体的相反方向移动。在关键主体平面KSP后面的物体在给定的运动中会有较少的视差。The horizontal image translation (HIT) sets the key subject plane KSP to the plane of the screen (the first or near plane) where the scene is generated. This step also sets the motion of objects, such as the bushes B of the near plane NP (third or near plane) and the trees T of the far plane FP (second or far plane) relative to each other. Objects in front of the key subject KS or key subject plane KSP move in one direction (left to right or right to left), while objects behind the key subject KS or key subject plane KSP move in the opposite direction to the objects in front. Objects behind the key subject plane KSP will have less parallax for a given motion.
在图11、图11A和图11B的示例中，每个层1100包括场景S的输入文件图像的主要图像元素，例如分别来自数字图像捕获装置831至834(n个装置)的图像或帧1101、1102、1103和1104。图像获取应用624执行一个过程，以平移图像或帧1101、1102、1103和1104，图像或帧1101、1102、1103和1104是重叠的并且从主轴1112偏移了计算出的视差值，(水平图像平移(HIT)。视差线1107表示关键主体KS点1109.1至1109.4从主轴1112的线性位移。优选地是视差线1107之间的Δ1120代表视差1120的线性量，诸如前视差1120.2和后视差1120.1。In the example of FIG. 11 , FIG. 11A and FIG. 11B , each layer 1100 includes the main image elements of the input file image of the scene S, such as images or frames 1101, 1102, 1103 and 1104 from digital image capture devices 831 to 834 (n devices), respectively. The image acquisition application 624 performs a process to translate the images or frames 1101, 1102, 1103 and 1104, which are overlapped and offset from the main axis 1112 by the calculated disparity value, (Horizontal Image Translation (HIT). Disparity lines 1107 represent the linear displacement of key subject KS points 1109.1 to 1109.4 from the main axis 1112. Preferably, the Δ 1120 between the disparity lines 1107 represents the linear amount of disparity 1120, such as the front disparity 1120.2 and the rear disparity 1120.1.
根据像素数、像素密度和帧数的函数、以及最近点和最远点和如美国专利9,992,473、美国专利10,033,990和美国专利10,178,247设定的其他参数来计算视差、最小视差和最大视差，这些专利通过引用全部并入本文。The disparity, minimum disparity, and maximum disparity are calculated as a function of the number of pixels, pixel density, and frame number, as well as the closest and farthest points and other parameters as set forth in US Pat. No. 9,992,473, US Pat. No. 10,033,990, and US Pat. No. 10,178,247, which are incorporated herein by reference in their entirety.
在块或步骤1220B中，计算机系统10通过深度图应用程序624被配置为创建源图像、由数字图像捕获装置831至834(n个装置)捕获的场景S的多个图像的深度图并且通过算法来制成灰度图像。深度图是图像或图像通道，它包含与场景S中的物体、表面或点距离视点诸如数字图像捕获装置831至834(n个装置)的距离有关的信息。例如，这提供了更多的信息，因为体积、纹理和照明被更充分地定义。一旦生成了深度图1220B，那么视差就可以得到严格的控制。为此，计算机系统10可以将输出帧的数量限制为四个，而不转向深度图。如果我们使用来自深度图的四个或来自深度图的两个，我们就不会被中间的相机位置所限制。注意外部的数字图像捕获装置831和834被锁定在观察者或用户U观看显示器628的瞳孔间距离(IPD)中。In block or step 1220B, the computer system 10 is configured to create a source image, a depth map of multiple images of the scene S captured by the digital image capture devices 831 to 834 (n devices) and a grayscale image by an algorithm through a depth map application 624. A depth map is an image or image channel that contains information about the distance of objects, surfaces or points in the scene S from the viewpoint such as the digital image capture devices 831 to 834 (n devices). For example, this provides more information because the volume, texture and lighting are more fully defined. Once the depth map 1220B is generated, the parallax can be strictly controlled. To this end, the computer system 10 can limit the number of output frames to four without turning to the depth map. If we use four from the depth map or two from the depth map, we are not limited by the camera position in the middle. Note that the external digital image capture devices 831 and 834 are locked in the interpupillary distance (IPD) of the observer or user U viewing the display 628.
此外，计算机系统10通过关键主体、应用程序624可以基于源图像的深度图来识别关键主体KS。类似地，计算机系统10通过深度图应用程序624可以使用源图像的深度图来识别近平面NP、远平面FP、前景、最近的点和背景、最远的点，近平面NP可以是通过与镜头平面最接近的焦点(前景中的灌木B)的平面，远平面FP是通过最远的焦点(背景中的树T)的平面。In addition, the computer system 10, through the key subject, application 624, can identify the key subject KS based on the depth map of the source image. Similarly, the computer system 10, through the depth map application 624, can use the depth map of the source image to identify the near plane NP, the far plane FP, the foreground, the nearest point, and the background, the farthest point, the near plane NP can be the plane passing through the focus closest to the lens plane (the bush B in the foreground), and the far plane FP is the plane passing through the farthest focus (the tree T in the background).
计算机系统10通过深度图应用程序624可以限定用于场景的一系列2D图像中的每一个的两个或更多个平面，并且一个或更多个平面可以具有不同的深度估计。计算机系统10通过深度图应用程序624可以识别场景的一系列2D图像内的第一近侧平面，诸如关键主体平面KSP、和第二远侧平面，诸如近平面NP或远平面FP。The computer system 10, via the depth map application 624, may define two or more planes for each of the series of 2D images of the scene, and one or more of the planes may have different depth estimates. The computer system 10, via the depth map application 624, may identify a first near plane, such as a key subject plane KSP, and a second far plane, such as a near plane NP or a far plane FP, within the series of 2D images of the scene.
在块或步骤1220C中，计算机系统10经由插于中间应用程序624被配置成将具有RGB高分辨率颜色(DIFY或3D立体)的场景S的图像(n个)1101、1102、1103、1104(帧集1100)的模型或网格叠加在2D RGB高分辨率数字相机上。In block or step 1220C, the computer system 10 is configured via the plug-in intermediate application 624 to overlay a model or mesh of images(n) 1101, 1102, 1103, 1104 (frame set 1100) of a scene S having RGB high resolution color (DIFY or 3D stereo) on a 2D RGB high resolution digital camera.
在块或步骤1220D中，计算机系统10经由数据集操纵应用624可以用于从场景S的图像(n个)1101、1102、1103、1104(帧集1100)生成场景S的模型或网格。In block or step 1220D, the computer system 10 via the data set manipulation application 624 may be used to generate a model or mesh of the scene S from the images (n) 1101 , 1102 , 1103 , 1104 (frame set 1100 ) of the scene S.
在块或步骤1225中，计算机系统10通过帧建立程序624被配置成创建或生成帧，记录来自虚拟相机移动、旋转或弧形位置的场景S的图像(n个)1101、1102、1103、1104(帧集1100)的图像，诸如诸如帧之间0.5至1度的分离或移动，诸如-5、-4、-3、-2、-1、0、1、2、3、4、5；为DIFY代表场景S的3D彩色网格数据集1101、1102、1103、1104(帧集1100)生成视差；对于左右3D立体；例如3D彩色网格数据集的1102、1103图像和(DIFY)的1101、1102、1103、1104。计算机系统10通过关键主体、应用程序624可以建立偏移的增量，例如视图之间总偏移的一(1)度(通常在显示器628上偏移10-70像素)。这仅仅意味着围绕关键主体KS的360度完整传感器(捕获装置)旋转将具有360度视图，因此我们仅使用/需要视图1和视图2，对于3D立体为左和右；1102、1103图像数据集。假设旋转视差围绕关键主体(零视差点)盘旋，这为每个视图提供了1度的分离/视差。这将可能建立最小差异/视差，当传感器(图像捕获模块830)远离关键主体KS移动更远时，可以向上调整该最小差异/视差。In block or step 1225, the computer system 10 is configured to create or generate frames via the frame building program 624, recording images of the scene S from the virtual camera movement, rotation or arc position (n) 1101, 1102, 1103, 1104 (frame set 1100), such as such as 0.5 to 1 degree separation or movement between frames, such as -5, -4, -3, -2, -1, 0, 1, 2, 3, 4, 5; generating disparity for DIFY 3D color grid data set 1101, 1102, 1103, 1104 (frame set 1100) representing the scene S; for left and right 3D stereo; for example 1102, 1103 images of the 3D color grid data set and 1101, 1102, 1103, 1104 of (DIFY). The computer system 10, via the key subject, application 624 can establish increments of offset, such as one (1) degree of total offset between views (typically 10-70 pixels on the display 628). This simply means that a full sensor (capture device) rotation of 360 degrees around the key subject KS will have a 360 degree view, so we only use/need view 1 and view 2, left and right for 3D stereo; 1102, 1103 image data sets. Assuming rotational parallax is circling around the key subject (zero parallax point), this provides 1 degree of separation/parallax for each view. This will likely establish a minimum disparity/parallax that can be adjusted upward as the sensor (image capture module 830) moves farther away from the key subject KS.
在块或步骤1225A中，计算机系统10通过帧建立程序624被配置成输入或上传从计算机系统10外部捕获的源图像。In block or step 1225A, the computer system 10 is configured via the frame creation program 624 to import or upload a source image captured from outside the computer system 10.
在块或步骤1230中，利用计算机系统10通过水平和垂直帧DIF平移程序624可以被配置为水平地和竖直地对准或转换每个源图像，由数字图像捕获装置831至834(n个装置)捕获的场景S的多个图像需要尺寸图像格式(DIF)转换。DIF变换是一种几何移位，它不改变在源图像、由数字图像捕获装置831至834(n个装置)捕获的场景S的多个图像中的每个点所获得的信息，但可以被视为源图像、由数字图像捕获装置831至834(n个装置)捕获的场景S的多个图像中的所有其他点在笛卡尔空间中的移位(如图11中图示的)。作为全光函数，DIF变换由以下方程表示：In block or step 1230, the multiple images of the scene S captured by the digital image capture devices 831 to 834 (n devices) require a dimension image format (DIF) conversion using the computer system 10 through the horizontal and vertical frame DIF shift program 624. The DIF transform is a geometric shift that does not change the information obtained at each point in the source image, the multiple images of the scene S captured by the digital image capture devices 831 to 834 (n devices), but can be regarded as a shift in Cartesian space of all other points in the source image, the multiple images of the scene S captured by the digital image capture devices 831 to 834 (n devices) (as illustrated in Figure 11). As a plenoptic function, the DIF transform is represented by the following equation:
其中，Δu,v＝Δθ,φWhere Δu,v＝Δθ,φ
在数字图像源的情况下，几何移位对应于包含全光信息的像素的几何移位，然后DIF变换变成：In the case of a digital image source, the geometric shift corresponds to the geometric shift of the pixels containing the plenoptic information, and the DIF transform then becomes:
(Pixel)x，y＝(Pixel)x，y+Δx，y(Pixel)x，y ＝(Pixel)x，y +Δx，y
此外，计算机系统10通过水平和垂直帧DIF平移应用624也可以使用DIF变换对背景和或前景进行几何移位。背景和前景可以根据各自相对于源图像、由数字图像捕获装置831至834(n个装置)捕获的场景S的多个图像的深度图1220B识别的关键主体KS的深度进行几何移位。控制背景和前景相对于关键主体KS的几何移位控制了关键主体KS的运动视差。如所述的，关键主体KS相对背景或前景的明显相对运动为观察者提供了关于其相对距离的暗示。通过这种方式，运动视差被控制成聚焦所显示的场景中处于不同深度的对象，以匹配辐辏和立体视网膜的视差需求，从而更好地模拟自然观看条件。通过调整场景中关键主体KS的焦点以匹配其立体视网膜视差(眼内或瞳孔间距离宽度IPD(人类视觉系统的瞳孔之间的距离))，使用于眼球调节和辐辏的提示一致。In addition, the computer system 10 can also use DIF transformation to geometrically shift the background and/or foreground through the horizontal and vertical frame DIF translation application 624. The background and foreground can be geometrically shifted according to the depth of the key subject KS identified by the depth map 1220B of multiple images of the scene S captured by the digital image capture device 831 to 834 (n devices) relative to the source image. Controlling the geometric shift of the background and foreground relative to the key subject KS controls the motion parallax of the key subject KS. As described, the apparent relative motion of the key subject KS relative to the background or foreground provides the observer with a hint about its relative distance. In this way, the motion parallax is controlled to focus on objects at different depths in the displayed scene to match the parallax requirements of convergence and stereoscopic retina, thereby better simulating natural viewing conditions. By adjusting the focus of the key subject KS in the scene to match its stereoscopic retinal parallax (intraocular or interpupillary distance width IPD (distance between pupils of the human visual system)), the prompts for eyeball accommodation and convergence are consistent.
再次参照图4，观看显示器628上的DIFY、多维图像序列1010需要用户U的两个不同的眼睛动作。首先是眼睛将追踪显示器628上的多维图像序列1010中最近的项、点或物体(近平面NP)，由于图像或帧1101、1102、1103和1104是重叠的并且从主轴1112偏移了计算出的视差值(水平图像平移(HIT))，因此该图像将来回线性平移到静止的关键主体平面KSP。这种跟踪是通过眼球移动跟随运动而发生的。其次，由于任何点或物体相对于关键主体平面KSP，更具体地说，相对于关键主体KS点的平滑运动变化，眼睛会感知到深度。因此，DIFY是由一个机械步骤和两个眼睛功能组成。Referring again to FIG. 4 , viewing the DIFY, multi-dimensional image sequence 1010 on the display 628 requires two distinct eye actions of the user U. First, the eye will track the nearest item, point, or object (the near plane NP) in the multi-dimensional image sequence 1010 on the display 628, which will be linearly translated back and forth to the stationary key subject plane KSP because the images or frames 1101, 1102, 1103, and 1104 are overlapping and offset from the principal axis 1112 by the calculated disparity value (horizontal image translation (HIT)). This tracking occurs by following the motion with eye movements. Second, the eye will perceive depth due to the smooth motion changes of any point or object relative to the key subject plane KSP, and more specifically, relative to the key subject point KS. Thus, DIFY consists of one mechanical step and two eye functions.
一个机械步骤是平移帧，使关键主体KS点在所有帧上重叠。由于图像或帧1101、1102、1103和1104可以是重叠的并且从主轴1112偏移了计算出的视差值(水平图像平移(HIT))，因此来回线性平移到静止的关键主体平面KSP。眼睛跟随近平面NP物体的运动，相对于关键主体KS表现出最大的运动(眼睛旋转)。沿着关键主体平面KSP(平滑眼睛运动)的帧位置的差异，引入了双眼视差。关键主体KS以外的任何两点的比较也会产生深度(双眼视差)。关键主体平面KSP后面的点以与关键主体KS前面的点的相反方向移动。在关键主体KS平面前面或后面或跨越关键主体KS平面的两个点的比较显示深度。One mechanical step is to translate the frames so that the key subject KS point overlaps on all frames. Since the images or frames 1101, 1102, 1103 and 1104 can be overlapping and offset from the main axis 1112 by the calculated disparity value (horizontal image translation (HIT)), a linear translation back and forth is performed to the stationary key subject plane KSP. The eyes follow the movement of the near-plane NP objects that exhibit the greatest movement (eye rotation) relative to the key subject KS. The difference in frame position along the key subject plane KSP (smooth eye movement) introduces binocular disparity. Comparison of any two points outside the key subject KS also produces depth (binocular disparity). Points behind the key subject plane KSP move in the opposite direction to points in front of the key subject KS. Comparison of two points in front of or behind the key subject KS plane or across the key subject KS plane shows depth.
在块或步骤1235A中，计算机系统10通过回文应用626被配置为创建、生成或产生多维数字图像序列1010，以无缝回文循环将来自数字图像捕获装置831至834(n个装置)的场景S的图像(n)中的每个图像按顺序对准(按顺序对准)，例如按顺序显示如下循环：来自第一数字图像捕获装置831的第一数字图像、图像或帧1101(1)，来自第二数字图像捕获装置832捕获装置832的第二数字图像、图像或帧1102(2)，来自第三数字图像捕获装置833的第三数字图像、图像或帧1103(3)，来自第四数字图像捕获装置834的第四数字图像、图像或帧1104(4)。此外，另一个序列是如下循环：来自第一数字图像捕获装置831的第一数字图像、图像或帧1101(1)，来自第二数字图像捕获装置832捕获装置832的第二数字图像、图像或帧1102(2)，来自第三数字图像捕获装置833的第三数字图像、图像或帧1103(3)，来自第四数字图像捕获装置834的第四数字图像、图像或帧1104(4)，来自第四数字图像捕获装置834的第四数字图像、图像或帧1104(4)，来自第三数字图像捕获装置833的第三数字图像、图像或帧1103(3)，来自第二数字图像捕获装置832的第二数字图像、图像或帧1102(2)，来自第一数字图像捕获装置831的第一数字图像、图像或帧1101(1)-1、2、3、4、4、3、2、1(按顺序对准)。优选的序列是源图像、由数字图像捕获装置831至834(n个装置)捕获的场景S的多个图像遵循图像被捕获的相同顺序或次序，并且添加倒置或反向顺序，以创建无缝的回文循环。In block or step 1235A, the computer system 10 is configured to create, generate or produce a multi-dimensional digital image sequence 1010 via a palindrome application 626 to sequentially align (sequentially align) each image in an image (n) of a scene S from digital image capture devices 831 to 834 (n devices) in a seamless palindrome cycle, such as displaying the following cycle in sequence: a first digital image, image or frame 1101 (1) from a first digital image capture device 831, a second digital image, image or frame 1102 (2) from a second digital image capture device 832, a third digital image, image or frame 1103 (3) from a third digital image capture device 833, and a fourth digital image, image or frame 1104 (4) from a fourth digital image capture device 834. In addition, another sequence is the following cycle: a first digital image, image or frame 1101(1) from a first digital image capture device 831, a second digital image, image or frame 1102(2) from a second digital image capture device 832, a third digital image, image or frame 1103(3) from a third digital image capture device 833, a fourth digital image, image or frame 1104(4) from a fourth digital image capture device 834, a fourth digital image, image or frame 1104(4) from the fourth digital image capture device 834, a third digital image, image or frame 1103(3) from the third digital image capture device 833, a second digital image, image or frame 1102(2) from the second digital image capture device 832, a first digital image, image or frame 1101(1) from the first digital image capture device 831 - 1, 2, 3, 4, 4, 3, 2, 1 (aligned in sequence). The preferred sequence is that the source images, multiple images of scene S captured by digital image capture devices 831 to 834 (n devices) follow the same order or sequence in which the images were captured, with the inversion or reverse sequence added to create a seamless palindrome.
在此设想，也可以在此配置其他序列，包括但不限于1、2、3、4、3、2、1(按顺序对准)等。It is contemplated that other sequences may also be configured herein, including but not limited to 1, 2, 3, 4, 3, 2, 1 (aligned in sequence), etc.
在此设想，对第一近侧平面进行水平地和竖直地对准，例如来自数字图像捕获装置831至834(n个装置)的场景S的图像(n)的每个图像的关键主体平面KSP，并且基于第二远侧平面的深度估计来移位序列中每个后续图像帧的第二远侧平面，例如前景平面、近平面NP、或背景平面、远平面FP，以产生第二经修改的2D图像序列或第二经修改的2D图像对。It is envisioned herein that a first near plane, e.g., a key subject plane KSP, is horizontally and vertically aligned for each image of images (n) of a scene S from digital image capture devices 831 to 834 (n devices), and a second far plane, e.g., a foreground plane, near plane NP, or a background plane, far plane FP, is shifted for each subsequent image frame in the sequence based on a depth estimate of a second far plane to produce a second modified 2D image sequence or a second modified 2D image pair.
在块或步骤1235B中，计算机系统10通过间相应用626可以被配置成对每帧集1100的像素列进行间相(interphase)，具体为左图像1102和右图像1103，以生成与关键主体KS点对准的并在计算出的视差范围内的多维数字图像。如图16A所示，间相应用626可以被配置成从左图像1102和右图像1103获取像素的节段、带、行或列，诸如场景S的源图像对、左图像1102和右图像1103的列1602A，并在左图像1102-LE的列1602A和右图像1103-RE的列1602A之间交替对它们进行分层，并重新配置或将它们并排交错串联排列，诸如重复系列160A两列宽，并且对场景S的地形T的源图像对、左图像1102和右图像1103的所有层重复该配置以生成多维图像1010，其中列1602A的尺寸设置成一个像素1550宽。In block or step 1235B, the computer system 10, via the interphasing application 626, may be configured to interphase the pixel columns of each frame set 1100, specifically the left image 1102 and the right image 1103, to generate a multi-dimensional digital image that is point-aligned with the key subject KS and within the calculated disparity range. As shown in FIG. 16A , the inter-phase application 626 may be configured to take segments, strips, rows, or columns of pixels from the left image 1102 and the right image 1103, such as column 1602A of the source image pair, the left image 1102, and the right image 1103 of the scene S, and layer them alternately between column 1602A of the left image 1102-LE and column 1602A of the right image 1103-RE, and reconfigure or arrange them in a staggered series side by side, such as repeating series 160A two columns wide, and repeating this configuration for all layers of the source image pair, the left image 1102, and the right image 1103 of the terrain T of the scene S to generate a multi-dimensional image 1010, wherein the size of column 1602A is set to be one pixel 1550 wide.
在此设想，源图像、由捕获装置830捕获的场景S的多个图像与显示器628的尺寸和配置相匹配，与关键主体KS点对准并在计算出的视差范围内。It is contemplated here that the source images, multiple images of the scene S captured by the capture device 830, match the size and configuration of the display 628, are point-aligned with the key subject KS and are within the calculated parallax range.
现在给定多维图像序列1010，我们转而观察装置的观看侧。Now given the multi-dimensional image sequence 1010, we turn to the viewing side of the device.
在此设想，源图像、由数字图像捕获装置831至834(n个装置)捕获的场景S的多个图像与显示器628的尺寸和配置相匹配，与关键主体KS点对准并在计算出的视差范围内。It is contemplated here that the source images, multiple images of scene S captured by digital image capture devices 831 to 834 (n devices), match the size and configuration of display 628, are point-aligned with key subject KS and within the calculated parallax range.
在块或步骤1240中，计算机系统10通过图像编辑应用624被配置为裁剪、缩放、对准、增强或对来自捕获装置831至834(n个装置)的场景S的每个图像(n)执行编辑或编辑多维数字图像序列1010。In block or step 1240, the computer system 10 is configured via the image editing application 624 to crop, scale, align, enhance, or perform editing or edit the multi-dimensional digital image sequence 1010 on each image (n) of the scene S from the capture devices 831 to 834 (n devices).
此外，计算机系统10和编辑应用程序624可以使用户U执行帧增强、层丰富、动画、羽化(平滑)、(Photoshop或Acorn照片或图像工具)，使图像(n)平滑或填充在一起，或其他用于在显示器628上产生3D效果的软件技术。在此设想，计算机系统10(自动模式)、显示器628和应用程序624可以执行算法或一组步骤，以自动地或使自动执行关键主体KS点的像素、像素组的对准或编辑对准、裁剪、缩放、对准、增强或对由数字图像捕获装置831至834(n个装置)捕获的场景S的多个图像执行编辑或编辑多维数字图像序列1010。Additionally, the computer system 10 and editing application 624 may enable the user U to perform frame enhancement, layer enrichment, animation, feathering (smoothing), (Photoshop or Acorn photo or image tools), smoothing or filling images (n) together, or other software techniques for producing 3D effects on the display 628. It is contemplated that the computer system 10 (automatic mode), the display 628, and the application 624 may execute an algorithm or set of steps to automatically or automatically perform pixel, pixel group alignment or editing alignment of key subject KS points, cropping, scaling, aligning, enhancing, or performing editing or editing a multi-dimensional digital image sequence 1010 on multiple images of a scene S captured by digital image capture devices 831 to 834 (n devices).
替代地，在块或步骤1240中，利用计算机系统10(在手动模式下)、显示器628和应用程序624的设置，以至少部分地使用户U对准或编辑关键主体KS点的像素、像素组的对准、裁剪、缩放、对准、增强或对由数字图像捕获装置831至834(n个装置)捕获的场景S的多个图像执行编辑或编辑多维数字图像序列1010。Alternatively, in block or step 1240, settings of the computer system 10 (in manual mode), the display 628, and the application 624 are utilized to at least partially enable the user U to align or edit pixels of key subject KS points, align, crop, scale, align, enhance, or perform editing on multiple images of a scene S captured by digital image capture devices 831 to 834 (n devices) or edit a multi-dimensional digital image sequence 1010.
此外，用户U通过显示器628和编辑应用程序624可以设置或选择每一帧的速度(查看时间)和一直查看周期的数量，如图13所示。时间间隔可以分配给多维数字图像序列1010中的每一帧。此外，可以在步骤1240调整帧之间的时间间隔，以提供平滑的运动和多维数字图像序列1010的最佳3D视图。In addition, the user U, through the display 628 and the editing application 624, can set or select the speed (viewing time) of each frame and the number of viewing cycles, as shown in FIG13. A time interval can be assigned to each frame in the multi-dimensional digital image sequence 1010. In addition, the time interval between frames can be adjusted at step 1240 to provide smooth motion and optimal 3D view of the multi-dimensional digital image sequence 1010.
在此设想，计算机系统10、显示器628和应用程序624可以执行算法或一组步骤，以自动或手动编辑或应用效果到帧集1100。此外，计算机系统10和编辑应用程序206可以包括编辑，诸如帧增强、层丰富、羽化、(Photoshop或Acorn照片或图像工具)，以使图像(n个)平滑或填充在一起，以及用于产生3D效果以在显示器628上显示场景S的地形T的3D多维图像的其他软件技术。It is contemplated that the computer system 10, display 628, and application 624 may execute an algorithm or set of steps to automatically or manually edit or apply effects to the frame set 1100. In addition, the computer system 10 and editing application 206 may include editing such as frame enhancement, layer enrichment, feathering, (Photoshop or Acorn photo or image tools) to smooth or fill the image(s) together, and other software techniques for producing a 3D effect to display a 3D multi-dimensional image of the terrain T of the scene S on the display 628.
在块或步骤1250中，计算机系统10通过图像显示应用624被配置为使场景S的图像(n)对于不同尺寸的显示器628通过顺序回文循环在显示器628上显示场景S的多维数字图像序列1010。同样，场景S的多维数字图像序列1010、产生的3D图像序列可以作为DIF序列输出到显示器628。在此设想，计算机系统10、显示器628和应用程序624可以是响应性的，即计算机系统10可以执行指令以将场景S的每个图像(n)的尺寸设定为适合给定显示器628的尺寸。In block or step 1250, the computer system 10 is configured through the image display application 624 to display the multi-dimensional digital image sequence 1010 of the scene S on the display 628 through a sequential palindrome loop for the images (n) of the scene S for the displays 628 of different sizes. Likewise, the multi-dimensional digital image sequence 1010 of the scene S and the resulting 3D image sequence can be output as a DIF sequence to the display 628. It is contemplated that the computer system 10, the display 628, and the application 624 can be responsive, i.e., the computer system 10 can execute instructions to set the size of each image (n) of the scene S to the size appropriate for a given display 628.
此外，用户U可以选择返回到块或步骤1220，以在由数字图像捕获装置831至834(n个装置)捕获的场景S的每个源图像、多个图像中选择新的关键主体KS，并进行通过步骤1220至1250，以通过创建新的或第二顺序循环，在显示器628上查看新的关键主体KS的场景S的多维数字图像序列1010。In addition, user U may choose to return to block or step 1220 to select a new key subject KS in each source image, multiple images of the scene S captured by the digital image capture devices 831 to 834 (n devices), and proceed through steps 1220 to 1250 to view the multidimensional digital image sequence 1010 of the scene S of the new key subject KS on the display 628 by creating a new or second sequential loop.
现在给定多维图像序列1010，我们转而观察装置的观看侧。此外，在块或步骤735中，计算机系统10通过输出应用730(206)可以被配置成通过通信链路240和/或网络250、或5G计算机系统10和应用程序206为一个更多个用户系统220、222、224在显示器628上显示多维图像1010。Now given the multi-dimensional image sequence 1010, we turn to the viewing side of the device. In addition, in block or step 735, the computer system 10 through the output application 730 (206) can be configured to display the multi-dimensional image 1010 on the display 628 for one or more user systems 220, 222, 224 via the communication link 240 and/or network 250, or 5G computer system 10 and application 206.
对于3D立体，现在参照图15A，以示例而非限制的方式示出了显示器628的部件的示例性堆叠体的截面图。显示器628可以包括发射光的像素阵列或多个像素，诸如LCD面板部件1520堆，其具有电极(诸如前电极和后电极)、偏振器(诸如水平偏振器和竖直偏振器)、漫射器(诸如灰色漫射器、白色漫射器)、以及发射红R光、绿G光和蓝B光的背光源。此外，显示器628可以包括其他标准LCD用户U交互部件，诸如顶部玻璃盖1510，其中电容式触摸屏玻璃1512定位在顶部玻璃盖1510和LCD面板部件堆1520之间。在此设想，除了LCD之外，此处可以包括其他形式的显示器628，诸如LED、ELED、PDP、QLED和其他类型的显示技术。此外，显示器628可以包括透镜阵列，诸如优选定位在电容式触摸屏玻璃1512和LCD面板部件堆1520之间的双凸透镜1514，并且被配置成以如下方式弯曲或折射光：能够在显示器628上将一对交错的左右立体图像对显示为3D或多维数字图像1010、从而在显示器628上显示场景S的多维数字图像。透明粘合剂1530可以用于粘合堆中的元件，无论是用作水平粘合剂还是竖直粘合剂以保持堆中的多个元件。例如，为了在显示器628上产生3D视图或产生多维数字图像，需要将通过多个像素的1920x1200像素图像分成两半，即960x1200，并且多个像素中的任一半可以用于左图像和右图像。For 3D stereo, now with reference to FIG. 15A , a cross-sectional view of an exemplary stack of components of a display 628 is shown by way of example and not limitation. The display 628 may include an array of pixels or a plurality of pixels that emit light, such as a stack of LCD panel components 1520, having electrodes (such as front and rear electrodes), polarizers (such as horizontal polarizers and vertical polarizers), diffusers (such as gray diffusers, white diffusers), and a backlight source that emits red R light, green G light, and blue B light. In addition, the display 628 may include other standard LCD user U interactive components, such as a top glass cover 1510, wherein a capacitive touch screen glass 1512 is positioned between the top glass cover 1510 and the LCD panel component stack 1520. It is contemplated that, in addition to LCD, other forms of display 628 may be included here, such as LED, ELED, PDP, QLED, and other types of display technologies. In addition, the display 628 may include a lens array, such as a lenticular lens 1514 preferably positioned between the capacitive touch screen glass 1512 and the LCD panel component stack 1520, and configured to bend or refract light in such a manner that a pair of staggered left and right stereoscopic images can be displayed as a 3D or multi-dimensional digital image 1010 on the display 628, thereby displaying a multi-dimensional digital image of the scene S on the display 628. A transparent adhesive 1530 can be used to bond the elements in the stack, whether used as a horizontal adhesive or a vertical adhesive to hold multiple elements in the stack. For example, in order to generate a 3D view or generate a multi-dimensional digital image on the display 628, a 1920x1200 pixel image through multiple pixels needs to be divided into two halves, i.e., 960x1200, and any half of the multiple pixels can be used for the left image and the right image.
在此设想，透镜阵列可以包括弯曲或折射光的其他技术，诸如屏障屏(黑线)、双凸、抛物线、覆盖、波导、黑线等能够分离成左图像和右图像的技术。It is contemplated that the lens array may include other techniques for bending or refracting light such as barrier screens (black lines), biconvex, parabolas, covers, waveguides, black lines, etc. that can separate into left and right images.
在此进一步设想，当显示器628保持处于横向视图时双凸透镜514可以在竖直列中定向以在显示器628上产生多维数字图像。然而，当显示器628保持处于纵向视图时，3D效果不明显，从而能够使用同一显示器628进行2D和3D观看。It is further contemplated that the lenticular lenses 514 may be oriented in vertical columns when the display 628 is held in a landscape view to produce a multi-dimensional digital image on the display 628. However, when the display 628 is held in a portrait view, the 3D effect is not apparent, thereby enabling the same display 628 to be used for both 2D and 3D viewing.
在此进一步设想，平滑或其他图像降噪技术和前景主体聚焦可以用于柔化和增强显示器628上的3D视图或多维数字图像。It is further contemplated that smoothing or other image noise reduction techniques and foreground subject focusing may be used to soften and enhance the 3D view or multi-dimensional digital image on the display 628 .
现在参照图15B，通过示例而非限制的方式，示出了示例性折射元件诸如显示器628的双凸透镜1514的一个实施方式的代表性段或部段。双凸透镜1514的每个子元件是双凸透镜1514的弧形或弯曲或拱形段或部段1540(成形为弧)，双凸透镜514的每个子元件可以配置成具有重复的一系列梯形透镜段或多个子元件或折射元件。例如，每个弧形或弯曲或拱形段1540可以配置成具有双凸透镜1540的透镜峰1541并且尺寸设置成一个像素1550(发射红R光、绿G光和蓝B光)宽，诸如具有到透镜峰1541的分配的中心像素1550C。在此设想，中心像素1550C光作为中心光1560C穿过双凸透镜1540以向距像素1550或双凸透镜1514的梯形段或部段1540的观看距离VD的左眼LE和右眼RE提供显示器628上的图像的2D观看。此外，每个弧形或弯曲段1540可以被配置成具有成角度的部段，诸如透镜折射元件(诸如双凸透镜1540的透镜子元件1542(多个子元件))的透镜角A1，并且尺寸设置成一个像素宽，诸如具有分配给左透镜、具有角度A1的左透镜子元件1542L和具有角度A1的右透镜子元件1542R的左像素1550L和右像素1550R，例如分别折射穿过中心线CL的光的倾斜角和下降角。在此设想，像素1550L/R光穿过双凸透镜1540并弯曲或折射以提供左图像和右图像以实现显示器628上图像的3D观看；通过左像素1550L光通过左镜头角度1542L并弯曲或折射，诸如进入左透镜角度1542L的光弯曲或折射以穿过中心线CL到右R侧，左图像光1560L朝向左眼LE，并且右像素1550R光通过右透镜角度1542R并弯曲或折射，诸如进入右透镜角度1542R的光弯曲或折射以穿过中心线CL到左侧L，右图像光1560R朝向右眼RE，以在显示器628上产生多维数字图像。Referring now to FIG. 15B , by way of example and not limitation, representative segments or sections of one embodiment of an exemplary refractive element such as a lenticular lens 1514 of the display 628 are shown. Each sub-element of the lenticular lens 1514 is an arcuate or curved or arched segment or section 1540 (shaped as an arc) of the lenticular lens 1514, and each sub-element of the lenticular lens 514 can be configured to have a repeating series of trapezoidal lens segments or multiple sub-elements or refractive elements. For example, each arcuate or curved or arched segment 1540 can be configured to have a lens peak 1541 of the lenticular lens 1540 and be sized to be one pixel 1550 (emitting red R light, green G light, and blue B light) wide, such as having an assigned center pixel 1550C to the lens peak 1541. It is contemplated here that the center pixel 1550C light passes through the lenticular lens 1540 as center light 1560C to provide 2D viewing of the image on the display 628 to the left eye LE and the right eye RE at a viewing distance VD from the pixel 1550 or the trapezoidal segment or section 1540 of the lenticular lens 1514. In addition, each arcuate or curved segment 1540 can be configured to have an angled segment, such as lens angle A1 of a lens refractive element (such as lens sub-element 1542 (multiple sub-elements) of the lenticular lens 1540), and is sized to be one pixel wide, such as having left pixel 1550L and right pixel 1550R assigned to the left lens, left lens sub-element 1542L having angle A1 and right lens sub-element 1542R having angle A1, for example, to refract the light passing through the center line CL at an angle of inclination and a fall angle, respectively. It is envisioned here that pixel 1550L/R light passes through the dual convex lens 1540 and is bent or refracted to provide left and right images to enable 3D viewing of images on the display 628; light through the left pixel 1550L passes through the left lens angle 1542L and is bent or refracted, such as the light entering the left lens angle 1542L is bent or refracted to pass through the center line CL to the right R side, and the left image light 1560L is directed toward the left eye LE, and light through the right pixel 1550R passes through the right lens angle 1542R and is bent or refracted, such as the light entering the right lens angle 1542R is bent or refracted to pass through the center line CL to the left side L, and the right image light 1560R is directed toward the right eye RE to produce a multi-dimensional digital image on the display 628.
在此设想，左图像和右图像可以如美国专利9,992,473、美国专利10,033,990和美国专利10,178,247的图6.1至图6.3所阐述那样的产生，并且以电子方式传送至左像素550L和右像素550R。此外，2D图像可以以电子方式传送至中心像素550C。It is contemplated that left and right images may be generated as described in FIGS. 6.1-6.3 of US Pat. No. 9,992,473, US Pat. No. 10,033,990, and US Pat. No. 10,178,247, and electronically transmitted to left pixel 550L and right pixel 550R. Additionally, a 2D image may be electronically transmitted to center pixel 550C.
在该图中，每个透镜峰1541具有对应的左成角度的透镜和右成角度的透镜1542，诸如在透镜峰1541的任一侧并且各自分配一个像素的左成角度的透镜1542L和右成角度的透镜1542R，分别给其分配中心像素1550C、左像素1550L和右像素像素1550R。In this figure, each lens peak 1541 has corresponding left angled lenses and right angled lenses 1542, such as left angled lenses 1542L and right angled lenses 1542R on either side of the lens peak 1541 and each assigned a pixel, to which are assigned a center pixel 1550C, a left pixel 1550L, and a right pixel 1550R, respectively.
在该图中，观看角度A1是观看距离VD、显示器628的大小S的函数，其中A1＝2arctan(S/2VD)In this figure, the viewing angle A1 is a function of the viewing distance VD and the size S of the display 628, where A1 = 2arctan(S/2VD)
在一个实施方式中，每个像素可以由一组子像素配置。例如，为了在显示器628上产生多维数字图像，每个像素可以配置为LCD面板部件堆1520的一个或两个3x3子像素，LCD面板部件堆520发射穿过双凸透镜1540的段或部段以在显示器628上产生多维数字图像的一个或两个红色R光、一个或两个绿G光以及一个或两个蓝B光。红R光、绿G光和蓝B可以配置为三个水平子像素的竖直堆。In one embodiment, each pixel may be configured by a group of sub-pixels. For example, to produce a multi-dimensional digital image on the display 628, each pixel may be configured as one or two 3x3 sub-pixels of the LCD panel component stack 1520, which emits one or two red R lights, one or two green G lights, and one or two blue B lights through the segments or sections of the lenticular lens 1540 to produce a multi-dimensional digital image on the display 628. The red R light, green G light, and blue B light may be configured as a vertical stack of three horizontal sub-pixels.
在此认识到，梯形透镜1540弯曲或折射均匀地通过其中心C、左L侧和右R侧诸如左成角度的透镜1542L和右成角度的透镜1542R和透镜峰1541的光。It is recognized here that the trapezoidal lens 1540 bends or refracts light uniformly through its center C, left L side, and right R side such as left angled lens 1542L and right angled lens 1542R and lens peak 1541.
现在参照图15C，以示例而非限制的方式示出了显示器628的示例性双凸透镜1514的一个实施方式的原型段或部段。作为双凸透镜1514的梯形段或部段1540的每个段或多个子元件或折射元件可以被配置成具有重复的一系列梯形透镜段。例如，每个梯形段1540可以被配置成具有双凸透镜1540的透镜峰1541并且尺寸设置成一个或两个像素1550宽和平坦或直透镜诸如透镜谷1543并且尺寸设置成一个或两个像素1550宽(发射红R光、绿G光和蓝B光)。例如，透镜谷1543可以被分配中心像素1550C。在此设想，中心像素1550C光作为中心光1560C穿过双凸透镜1540以向距像素1550或双凸透镜1514的梯形段或部段1540的观看距离VD的左眼LE和右眼RE提供显示器628上的图像的2D观看。此外，每个梯形段1540可以被配置成具有成角度的部分，例如双凸透镜1540的透镜角度1542并且尺寸设置成一个或两个像素宽，诸如具有分别分配给左透镜角度1542L和右镜头角度1542R的左像素1550L和右像素1550R。在此设想，像素1550L/R光穿过双凸透镜1540并弯曲以提供左图像和右图像以实现显示器628上图像的3D观看；经由左像素1550L光通过左透镜角度1542L并弯曲或折射，诸如进入左透镜角度1542L的光弯曲或折射以穿过中心线CL至右R侧，左图像光1560L朝向左眼LE；并且右像素1550R光通过右透镜角度1542R并弯曲或折射，诸如进入右透镜角度1542R的光弯曲或折射以穿过中心线CL至左侧L，右图像光1560R朝向右眼RE；Referring now to FIG. 15C , a prototype segment or section of one embodiment of an exemplary lenticular lens 1514 of a display 628 is shown by way of example and not limitation. Each segment or multiple sub-elements or refractive elements as a trapezoidal segment or section 1540 of a lenticular lens 1514 can be configured to have a repeating series of trapezoidal lens segments. For example, each trapezoidal segment 1540 can be configured to have a lens peak 1541 of a lenticular lens 1540 and sized to be one or two pixels 1550 wide and a flat or straight lens such as a lens valley 1543 and sized to be one or two pixels 1550 wide (emitting red R light, green G light, and blue B light). For example, a lens valley 1543 can be assigned a central pixel 1550C. It is envisioned here that the central pixel 1550C light passes through the lenticular lens 1540 as a central light 1560C to provide 2D viewing of an image on the display 628 to the left eye LE and the right eye RE at a viewing distance VD from the pixel 1550 or the trapezoidal segment or section 1540 of the lenticular lens 1514. In addition, each trapezoidal segment 1540 can be configured to have an angled portion, such as lens angle 1542 of the lenticular lens 1540 and be sized to be one or two pixels wide, such as having left pixel 1550L and right pixel 1550R assigned to left lens angle 1542L and right lens angle 1542R, respectively. It is contemplated that pixel 1550L/R light passes through the lenticular lens 1540 and is bent to provide a left image and a right image to enable 3D viewing of images on the display 628; via left pixel 1550L light passes through left lens angle 1542L and is bent or refracted, such as light entering left lens angle 1542L is bent or refracted to pass through center line CL to the right R side, left image light 1560L is directed toward the left eye LE; and right pixel 1550R light passes through right lens angle 1542R and is bent or refracted, such as light entering right lens angle 1542R is bent or refracted to pass through center line CL to the left side L, right image light 1560R is directed toward the right eye RE;
在此设想，透镜角度1542的角度A1是像素1550大小、显示器628的部件的堆叠体、双凸透镜514的折射特性以及左眼LE和右眼RE距像素1550的距离——观看距离VD的函数。It is contemplated here that angle A1 of lens angle 1542 is a function of pixel 1550 size, the stack-up of components of display 628, the refractive properties of lenticular lens 514, and the distance of left eye LE and right eye RE from pixel 1550 - viewing distance VD.
在该图15C中，观看角度A1是观看距离VD、显示器628的大小S的函数，其中A1＝2arctan(S/2VD)。In FIG. 15C , the viewing angle A1 is a function of the viewing distance VD and the size S of the display 628 , where A1 = 2arctan(S/2VD).
现在参照图15D，以示例而非限制的方式示出了显示器628的示例性双凸透镜1514的一个实施方式的代表性段或部段。作为双凸透镜1514的抛物线或圆顶形段或部段1540A(抛物面透镜或圆顶透镜，圆顶形)的每个段或多个子元件或折射元件可以被配置成具有重复的一系列圆顶形、弯曲、半圆形透镜段。例如，每个圆顶段1540A可以配置成具有双凸透镜540的透镜峰1541并且尺寸设置成一个或两个像素1550(发射红R光、绿G光和蓝B光)宽，诸如具有到透镜峰1541的分配的中心像素1550C。在此设想，中心像素1550C光作为中心光560C穿过双凸透镜540以向距像素1550或双凸透镜1514的梯形段或部段1540的观看距离VD的左眼LE和右眼RE提供显示器628上的图像的2D观看。此外，每个梯形段1540可以被配置成具有成角度的部分，例如双凸透镜1540的透镜角度1542并且尺寸设置成一个像素宽，诸如具有分别分配给左透镜角度1542L和右镜头角度1542R的左像素1550L和右像素1550R。在此设想，像素1550L/R光穿过双凸透镜1540并弯曲以提供左图像和右图像以实现显示器628上图像的3D观看；经由左像素1550L光通过左镜头角度1542L并弯曲或折射，诸如进入左透镜角度1542L的光弯曲或折射以穿过中心线CL到右R侧，左图像光1560L朝向左眼LE，并且右像素1550R光通过右透镜角度1542R并弯曲或折射，诸如进入右透镜角度1542R的光弯曲或折射以穿过中心线CL到左侧L，右图像光1560R朝向右眼RE，以在显示器628上产生多维数字图像。Referring now to FIG. 15D , representative segments or sections of one embodiment of an exemplary lenticular lens 1514 of the display 628 are shown by way of example and not limitation. Each segment or multiple sub-elements or refractive elements that are parabolic or dome-shaped segments or sections 1540A (parabolic lens or dome lens, dome-shaped) of the lenticular lens 1514 can be configured to have a repeating series of dome-shaped, curved, semicircular lens segments. For example, each dome segment 1540A can be configured to have a lens peak 1541 of the lenticular lens 540 and be sized to be one or two pixels 1550 (emitting red R light, green G light, and blue B light) wide, such as having a central pixel 1550C assigned to the lens peak 1541. It is contemplated here that the center pixel 1550C light passes through the lenticular lens 540 as center light 560C to provide 2D viewing of the image on the display 628 to the left eye LE and the right eye RE at a viewing distance VD from the pixel 1550 or the trapezoidal segment or section 1540 of the lenticular lens 1514. In addition, each trapezoidal segment 1540 can be configured to have an angled portion, such as the lens angle 1542 of the lenticular lens 1540 and be sized to be one pixel wide, such as having a left pixel 1550L and a right pixel 1550R assigned to a left lens angle 1542L and a right lens angle 1542R, respectively. It is envisioned here that pixel 1550L/R light passes through the dual convex lens 1540 and is bent to provide left and right images to enable 3D viewing of images on the display 628; light via the left pixel 1550L passes through the left lens angle 1542L and is bent or refracted, such as light entering the left lens angle 1542L is bent or refracted to pass through the center line CL to the right R side, and the left image light 1560L is directed toward the left eye LE, and light via the right pixel 1550R passes through the right lens angle 1542R and is bent or refracted, such as light entering the right lens angle 1542R is bent or refracted to pass through the center line CL to the left side L, and the right image light 1560R is directed toward the right eye RE to produce a multi-dimensional digital image on the display 628.
在此认识到，圆顶形透镜1540B弯曲或折射几乎均匀地通过其中心C、左L侧和右R侧的光。It is recognized here that the dome-shaped lens 1540B bends or refracts light passing through its center C, left L side, and right R side almost uniformly.
在此认识到，示例性双凸透镜1514的一个实施方式的代表性段或部段可以配置成多种其他形状和尺寸。It is recognized that the representative segments or sections of one embodiment of the exemplary lenticular lens 1514 may be configured in a variety of other shapes and sizes.
此外，为了在同一显示器628上同时实现最高质量的二维(2D)图像观看和多维数字图像观看，可以在显示器628上的多维数字图像观看期间使用交替黑线或视差屏障(交替)的数字形式，而无需将双凸透镜1514添加到显示器628的堆叠体，然后在显示器628上的二维(2D)图像观看期间可以禁用交替黑线或视差屏障(交替)的数字形式的数字形式。In addition, in order to achieve the highest quality two-dimensional (2D) image viewing and multi-dimensional digital image viewing on the same display 628, a digital form of alternating black lines or a parallax barrier (alternating) can be used during multi-dimensional digital image viewing on the display 628 without adding a double convex lens 1514 to the stack of the display 628, and then the digital form of alternating black lines or a parallax barrier (alternating) can be disabled during two-dimensional (2D) image viewing on the display 628.
视差屏障是放置在图像源诸如液晶显示器前面的装置，以允许其显示立体或多视觉(multiscopic)图像，而无需观看者佩戴3D眼镜。放置在普通LCD的前面，它由具有一系列精确间隔的狭的不透明层组成，从而允许每只眼睛看到一组不同的像素，从而通过视差创造深度感。数字视差屏障是图像源诸如液晶显示器(像素)前面的一系列交替黑线，以允许其显示立体或多视觉图像。此外，面部跟踪软件功能可以用于根据用户眼睛的位置调整像素和屏障狭缝的相对位置，从而允许用户从广泛的位置体验3D。KeehoonHong、Soon-giPark、Jisoo Hong、Byoungho Lee的书籍Design and Implementation ofAutostereoscopic Displays(自动立体显示器的设计与实现)通过引用并入本文。A parallax barrier is a device placed in front of an image source such as a liquid crystal display to allow it to display stereoscopic or multi-vision images without the viewer having to wear 3D glasses. Placed in front of an ordinary LCD, it consists of a series of narrow opaque layers with precise spacing, allowing each eye to see a different set of pixels, thereby creating a sense of depth through parallax. A digital parallax barrier is a series of alternating black lines in front of an image source such as a liquid crystal display (pixel) to allow it to display stereoscopic or multi-vision images. In addition, facial tracking software functions can be used to adjust the relative position of the pixels and the barrier slits according to the position of the user's eyes, allowing the user to experience 3D from a wide range of positions. The book Design and Implementation of Autostereoscopic Displays by Keehoon Hong, Soon-gi Park, Jisoo Hong, and Byoungho Lee is incorporated herein by reference.
在此设想，视差和关键主体KS参考点计算可以针对虚拟相机位置之间的距离、间相间距、显示器628与用户U的距离、双凸透镜1514配置(透镜角度A1、1542、透镜每毫米和毫米深度)来制定阵列的)，透镜角度1542作为显示器628组件堆叠的函数，双凸透镜1514的折射特性，以及左眼LE和右眼RE距像素1550的距离，观看距离VD，虚拟相机之间的距离位置(瞳孔间距IPD)等，以产生与观看设备或其他观看功能相关的数字多维图像，例如屏障屏幕(黑线)、双凸透镜、抛物线、覆盖、波导、黑线等在LED或OLED、LCD、OLED及其组合或其他观看设备中具有集成LCD层。It is contemplated herein that parallax and key subject KS reference point calculations may be formulated for the distance between virtual camera positions, inter-phase spacing, distance of the display 628 from the user U, lenticular lens 1514 configuration (lens angles A1, 1542, lens per mm and mm depth of the array), lens angle 1542 as a function of display 628 component stacking, refractive properties of the lenticular lens 1514, and distances of the left eye LE and right eye RE from the pixel 1550, viewing distance VD, distance positions between virtual cameras (interpupillary distance IPD), etc., to produce a digital multi-dimensional image associated with a viewing device or other viewing function, such as a barrier screen (black line), lenticular lens, parabola, overlay, waveguide, black line, etc. with an integrated LCD layer in an LED or OLED, LCD, OLED and combinations thereof, or other viewing device.
通过引用并入本文的是其他显示技术等的Jason Geng的名称为Three-Dimensional Display Technology(三维显示技术)，第1-80页的论文，其可以用于产生通过引用并入的本文显示器628。Incorporated herein by reference is, among other display technologies, a paper by Jason Geng entitled Three-Dimensional Display Technology, pages 1-80, which may be used to produce the display 628 incorporated herein by reference.
在此设想，双凸透镜514的每毫米或每英寸透镜数量由显示器628的每英寸像素数确定。It is contemplated here that the number of lenses per millimeter or per inch of the lenticular lens 514 is determined by the number of pixels per inch of the display 628.
在此设想，在此设想其他角度A1；像素1550C、1550L、1550R距透镜1540的距离(大约0.5mm)；以及用户U从用户的眼睛距智能装置显示器628的观看距离(大约十五(15)英寸))，并且人眼之间的平均人瞳孔间间距(大约2.5英寸)可以被分解或计算以产生数字多维图像。角度和间距的控制规则确保显示器628上的观看图像在观看装置的舒适区内以产生数字多维图像，参见下面的图5、图6、图11。It is contemplated here that other angles A1; the distances of pixels 1550C, 1550L, 1550R from the lens 1540 (approximately 0.5 mm); and the viewing distance of the user U from the user's eyes to the smart device display 628 (approximately fifteen (15) inches)), and the average human interpupillary distance between human eyes (approximately 2.5 inches) can be decomposed or calculated to generate a digital multi-dimensional image. The control rules for angles and distances ensure that the viewing image on the display 628 is within the comfort zone of the viewing device to generate a digital multi-dimensional image, see Figures 5, 6, and 11 below.
在此认识到，可以基于用户U眼——左眼LE和右眼RE与像素550诸如像素1550C、1550L、1550R之间的观看距离VD来计算和设置透镜1541的角度A1，该观看距离是保持显示器628距用户U眼睛诸如十(10)英寸到手臂/手腕的长度、或者更优选地在大约十五(15)英寸至二十四(24)英寸之间、并且最优选地在大约十五(15)英寸的舒适距离。It is recognized here that the angle A1 of the lens 1541 can be calculated and set based on the viewing distance VD between the user's U eyes - the left eye LE and the right eye RE and the pixels 550 such as pixels 1550C, 1550L, 1550R, which viewing distance is a comfortable distance that keeps the display 628 away from the user's U eyes, such as ten (10) inches to the length of the arm/wrist, or more preferably between approximately fifteen (15) inches to twenty-four (24) inches, and most preferably at approximately fifteen (15) inches.
在使用中，用户U将显示器628朝向和远离用户的眼睛移动，直到数字多维图像出现在用户面前，该移动考虑到用户的U实际瞳孔间距离IPD间距，并匹配用户的视觉系统(近方差异和远方差异)、作为来自虚拟相机位置(瞳孔间距离IPD)之间的距离的左右交错图像的宽度位置的函数、场景S的数字图像(n个)中的每个数字图像中关键主体KS深度(关键主体KS算法)、两个图像(左图像和右图像)的关于关键主体KS的水平图像平移算法、两个图像(左图像和右图像)的关于关键对体KS的间相算法、角度A1、像素1550距透镜1540的距离(像素-透镜距离(PLD)大约为0.5mm))、以及透镜阵列诸如梯形透镜1540的折射特性，所有这些都被考虑以产生数字多维图像供用户U观看显示器628。首先已知的元素是像素数1550和图像数、两个图像、虚拟相机位置之间的距离或(瞳孔间距离IPD)。在瞳孔间距离IPD处或附近捕获的图像与人类视觉系统相匹配，这简化了数学运算，最大限度地减少了两个图像之间的串扰、模糊性、图像移动，以产生可在显示器628上观看的数字多维图像。In use, the user U moves the display 628 toward and away from the user's eyes until a digital multi-dimensional image appears in front of the user, the movement taking into account the user's U actual interpupillary distance IPD spacing and matching the user's visual system (near disparity and far disparity), the width position of the left and right interlaced images as a function of the distance between the virtual camera positions (interpupillary distance IPD), the depth of the key subject KS in each of the digital images (n) of the scene S (key subject KS algorithm), the horizontal image translation algorithm of the two images (left and right) about the key subject KS, the inter-phase algorithm of the two images (left and right) about the key pair KS, the angle A1, the distance of the pixel 1550 from the lens 1540 (the pixel-lens distance (PLD) is approximately 0.5 mm)), and the refractive properties of the lens array such as the trapezoidal lens 1540, all of which are taken into account to produce the digital multi-dimensional image for the user U to view the display 628. The first known elements are the number of pixels 1550 and the number of images, the two images, the distance between the virtual camera positions or (interpupillary distance IPD). Images captured at or near the interpupillary distance IPD are matched to the human visual system, which simplifies the mathematical operations and minimizes crosstalk, blurriness, and image movement between the two images to produce a digital multi-dimensional image that can be viewed on the display 628.
在此进一步设想，梯形透镜1540可以由聚苯乙烯、聚碳酸酯或其他透明材料或类似材料形成，因为这些材料提供多种形式和形状，可以制造成不同的形状和大小，并且提供重量降低的强度；然而，可以使用其他合适的材料等，只要这种材料具有透明性并且可机加工或可成型，以满足本文所述的目的以产生左右立体图像和指定的折射率。在此进一步设想，梯形透镜1541可以配置有每毫米4.5个双凸透镜和大约0.33mm的深度。It is further contemplated that the trapezoidal lens 1540 may be formed of polystyrene, polycarbonate or other transparent material or similar materials, as these materials provide a variety of forms and shapes, can be manufactured into different shapes and sizes, and provide strength with reduced weight; however, other suitable materials, etc. may be used, as long as such materials are transparent and can be machined or formed to meet the purposes described herein to produce left and right stereo images and the specified refractive index. It is further contemplated that the trapezoidal lens 1541 may be configured with 4.5 biconvex lenses per millimeter and a depth of approximately 0.33 mm.
DIFY，在块或步骤1250中，计算机系统10通过图像显示应用624被配置成使地形T的帧集1100对于不同尺寸的显示器628通过顺序回文循环在显示器628上显示多维数字图像序列1010。同样，场景S的多维数字图像序列1010、产生的3D图像序列可以作为DIF序列或.MPO文件输出到显示器628。在此设想，计算机系统10、显示器628和应用程序624可以是响应性的，即计算机系统10可以执行指令以将场景S的每个图像(n)的尺寸设定为适合给定显示器628的尺寸。DIFY, in block or step 1250, the computer system 10 is configured through the image display application 624 to display the multi-dimensional digital image sequence 1010 on the display 628 through a sequential palindrome loop for the frames 1100 of the terrain T for the displays 628 of different sizes. Similarly, the multi-dimensional digital image sequence 1010 of the scene S, the resulting 3D image sequence, can be output to the display 628 as a DIF sequence or a .MPO file. It is contemplated that the computer system 10, the display 628, and the application 624 can be responsive, that is, the computer system 10 can execute instructions to set the size of each image (n) of the scene S to the size suitable for a given display 628.
在块或步骤1250中，显示器628上的多维图像序列1010利用来自捕获装置831至834(n个装置)的场景S的图像(n)中的每个图像的物体相对于关键主体平面KSP的位置差异，该差异在序列中的图像之间引入视差，以在显示器628上显示多维图像序列1010，以使用户U在块或步骤1250中查看显示器628的多维图像序列1010。In block or step 1250, the multidimensional image sequence 1010 on the display 628 utilizes the position difference of objects in each image of the images (n) of the scene S from the capture devices 831 to 834 (n devices) relative to the key subject plane KSP, which introduces parallax between the images in the sequence to display the multidimensional image sequence 1010 on the display 628 so that the user U views the multidimensional image sequence 1010 on the display 628 in block or step 1250.
此外，在块或步骤1250中，计算机系统10通过输出应用624可以被配置为通过通信链路740和/或网络750、或5G计算机系统10和应用程序624为一个更多个用户系统720、722、724在显示器628上显示多维图像序列1010。Additionally, in block or step 1250 , the computer system 10 via the output application 624 may be configured to display the multidimensional image sequence 1010 on a display 628 for one or more user systems 720 , 722 , 724 via a communication link 740 and/or network 750 , or a 5G computer system 10 and application 624 .
3D立体，在块或步骤1250中，计算机系统10通过输出应用624可以被配置成在显示器628上显示多维图像1010。多维图像1010可以通过左和右像素1102L/1103R显示，光通过双凸透镜1540并弯曲或折射以在显示器628上向左眼LE和右眼RE提供多维图像1010的3D观看，距像素1550观看距离VD。3D Stereoscopic, in block or step 1250, the computer system 10 via the output application 624 may be configured to display the multi-dimensional image 1010 on the display 628. The multi-dimensional image 1010 may be displayed via the left and right pixels 1102L/1103R, with light passing through the lenticular lens 1540 and being bent or refracted to provide 3D viewing of the multi-dimensional image 1010 to the left eye LE and the right eye RE on the display 628, viewing distance VD from the pixel 1550.
在块或步骤1250中，利用计算机系统10、显示器628和应用程序624设置来配置来自场景S的地形T的帧集1100的场景S的每个图像(n个)(L&R段)，同时关键主体针对双眼视差在图像之间对准，用于在显示器208上显示/观看/保存多维数字主图像1010，其中来自虚拟相机的场景S的每个图像(n个)的位置差异相对于关键主体KS平面引入(左和右)双眼视差以在显示器208上显示多维数字图像1010，以使用户U能够在块或步骤1250中在显示器208上观看多维数字图像。In block or step 1250, each image (n) of the scene S (L&R segments) of the frame set 1100 of the terrain T of the scene S is configured using the computer system 10, the display 628 and the application 624 settings, while the key subject is aligned between the images for binocular parallax, for displaying/viewing/saving the multi-dimensional digital master image 1010 on the display 208, wherein the position difference of each image (n) of the scene S from the virtual camera introduces (left and right) binocular parallax relative to the key subject KS plane to display the multi-dimensional digital image 1010 on the display 208, so as to enable the user U to view the multi-dimensional digital image on the display 208 in block or step 1250.
此外，用户U可以选择返回到块或步骤1220，以在每个源图像、场景S的地形T的帧集1100中选择新的关键主体KS，并进行通过步骤1220至1250，以通过创建新的或第二顺序循环，在显示器628上查看新的关键主体KS的场景S的多维数字图像序列1010。In addition, the user U may choose to return to block or step 1220 to select a new key subject KS in each source image, frame set 1100 of the terrain T of the scene S, and proceed through steps 1220 to 1250 to view the multidimensional digital image sequence 1010 of the scene S of the new key subject KS on the display 628 by creating a new or second sequential loop.
显示器628可以包括显示装置(例如，无论是在智能手机、PDA、监视器、电视、平板电脑或能够投射像素格式的信息的其他观看装置上实现的观看屏幕)或打印机(例如，消费者打印机、商店亭、专用打印机或其他硬拷贝装置)，以在例如光栅或其他物理观看材料上打印多维数字主图像。Display 628 may include a display device (e.g., a viewing screen whether implemented on a smartphone, PDA, monitor, television, tablet, or other viewing device capable of projecting information in a pixel format) or a printer (e.g., a consumer printer, store kiosk, specialty printer, or other hard copy device) to print the multi-dimensional digital master image on, for example, a lenticular or other physical viewing material.
在此认识到，步骤1220至1240可以由计算机系统10通过图像操纵应用626利用不同的和单独定位的计算机系统10，诸如一个或更多个用户系统720、722、724和应用程序626执行此处的步骤。例如，使用远离图像捕获系统并且远离图像浏览系统的图像处理系统，可以通过计算机系统10或服务器760和应用程序624远离场景S执行步骤1220至1240，并且通过通信链路740和/或网络750或者通过无线网络例如5G计算机系统10和应用程序626通过更多个用户系统720、722、724在用户系统720、722、724与应用程序626之间进行通信。在此，计算机系统1通过图像操纵应用624可以操纵24个设置以配置来自虚拟相机的场景S的场景S的每个图像(n)(L&R段)以生成与关键主体KS点对准的多维数字图像序列1010并且通过通信链路740和/或网络750或通过无线网络诸如5G计算机系统10或服务器760和应用程序624向一个或更多个用户系统720、722、724传输以显示多维数字图像/序列1010。It is recognized herein that steps 1220-1240 may be performed by computer system 10 via image manipulation application 626 utilizing different and separately located computer systems 10, such as one or more user systems 720, 722, 724 and application 626. For example, using an image processing system remote from the image capture system and remote from the image browsing system, steps 1220-1240 may be performed remote from scene S by computer system 10 or server 760 and application 624, and communication may be performed between user systems 720, 722, 724 and application 626 via more user systems 720, 722, 724 via communication link 740 and/or network 750 or via a wireless network such as 5G computer system 10 and application 626. Here, the computer system 1 through the image manipulation application 624 can manipulate 24 settings to configure each image (n) (L&R segments) of the scene S from the virtual camera to generate a multi-dimensional digital image sequence 1010 aligned with the key subject KS point and transmit to one or more user systems 720, 722, 724 through the communication link 740 and/or network 750 or through a wireless network such as a 5G computer system 10 or server 760 and application 624 to display the multi-dimensional digital image/sequence 1010.
此外，在此认识到，步骤1220至1240可以由计算机系统10通过图像操纵应用624利用定位在车辆上的不同的和单独定位的计算机系统10来执行。例如，使用远离图像捕获系统的图像处理系统，通过计算机系统10和应用程序624来执行步骤1220至1240，计算机系统10可以操纵24个设置来配置来自捕获装置830的场景S的场景S的每个图像(n)(L&R段)图像，以生成与关键主体KS点对准的多维数字图像/序列1010。在此，计算机系统10通过图像操纵应用626可以利用多维图像/序列1010来导航运载工具V通过场景S的地形T。替代地，计算机系统10通过图像操纵应用626可以使远离车辆V的用户U使用多维图像/序列1010以导航运载工具V通过场景S的地形T。Furthermore, it is recognized herein that steps 1220 to 1240 may be performed by the computer system 10 via the image manipulation application 624 utilizing a different and separately located computer system 10 located on the vehicle. For example, using an image processing system remote from the image capture system, the computer system 10 may manipulate the 24 settings to configure each image(n) (L&R segments) image of the scene S from the capture device 830 to generate a multi-dimensional digital image/sequence 1010 that is point-aligned with the key subject KS. Here, the computer system 10 via the image manipulation application 626 may utilize the multi-dimensional image/sequence 1010 to navigate the vehicle V through the terrain T of the scene S. Alternatively, the computer system 10 via the image manipulation application 626 may enable a user U remote from the vehicle V to use the multi-dimensional image/sequence 1010 to navigate the vehicle V through the terrain T of the scene S.
在此设想，计算机系统10通过输出应用624可以被配置为使多维图像序列1010在显示器628上显示，以使多个用户U在块或步骤1250中查看显示器628上的多维图像序列1010直播或作为重播/转播。It is contemplated here that the computer system 10 via the output application 624 may be configured to cause the multi-dimensional image sequence 1010 to be displayed on the display 628 , such that a plurality of users U view the multi-dimensional image sequence 1010 on the display 628 live or as a replay/rebroadcast in block or step 1250 .
在此认识到，步骤1250可以由计算机系统10通过输出应用624利用不同的和单独定位的计算机系统10来执行，诸如一个或更多个用户系统720、722、724和应用程序624执行此处的步骤。例如，使用远离场景S的输出或图像查看系统通过计算机系统10和应用程序624并且通过通信链路740和/或网络750或者通过无线网络例如5G计算机系统10和应用程序624通过更多个用户系统720、722、724在用户系统720、722、724与应用程序626之间进行通信。此处，计算机系统10输出应用624可以接收多个经操纵的场景S的两个数字图像，并通过通信链路740和/或网络750、或通过无线网络诸如5G计算机系统10和应用程序624向一个更多个用户系统720、722、724显示多维图像序列1010。It is recognized here that step 1250 may be performed by the computer system 10 through the output application 624 utilizing different and separately located computer systems 10, such as one or more user systems 720, 722, 724 and application 624 to perform the steps herein. For example, an output or image viewing system remote from the scene S is used to communicate between the user systems 720, 722, 724 and application 626 through the computer system 10 and application 624 and through more user systems 720, 722, 724 through a communication link 740 and/or network 750 or through a wireless network such as a 5G computer system 10 and application 624. Here, the computer system 10 output application 624 may receive two digital images of a plurality of manipulated scenes S and display the multi-dimensional image sequence 1010 to one or more user systems 720, 722, 724 through a communication link 740 and/or network 750 or through a wireless network such as a 5G computer system 10 and application 624.
此外，通过通信链路740和/或网络750、无线诸如5G第二计算机系统10和应用程序624可以将相对于关键主体平面KSP配置的场景S的图像集(n)作为多维图像序列1010传输到显示器628上，以使多个用户U在块或步骤1250中查看显示器628上的多维图像序列1010直播或作为重播/转播。In addition, via the communication link 740 and/or network 750, wireless such as 5G, the second computer system 10 and the application 624 can transmit the image set (n) of the scene S configured relative to the key subject plane KSP as a multidimensional image sequence 1010 to the display 628, so that multiple users U can view the multidimensional image sequence 1010 on the display 628 live or as a replay/rebroadcast in block or step 1250.
现在参照图13，通过示例而非限制的方式，图示出了触摸屏显示器628，使用户U能够选择计算机系统10的摄影选项。第一示例性选项可以是DIFY捕获，其中用户U可以指定或选择数字图像速度设置1302，其中用户U可以增加或减少数字图像在显示器628多维图像/序列1010上的顺序显示的回放速度或帧(图像)/秒。此外，用户U可以指定或选择数字图像的循环数或重复数1304，以设置场景S的多个2D图像1000的图像(n)的循环数，其中场景S的多个2D图像1000中的图像(n)以顺序显示在显示器628上，与图11类似。仍然进一步地，用户U可以指定或选择用于回放的数字图像序列或回文序列的回放顺序1306，以设置场景S的多维图像/序列1010的图像(n)的显示顺序。图像的定时顺序显示通过运动追逐比效应产生适当的双眼视差。在此设想计算机系统10和应用程序624可以利用在此的默认或自动设置。Referring now to FIG. 13 , by way of example and not limitation, a touch screen display 628 is illustrated that enables a user U to select photography options for the computer system 10 . A first exemplary option may be a DIFY capture, wherein the user U may specify or select a digital image speed setting 1302, wherein the user U may increase or decrease the playback speed or frames (images)/second of the sequential display of digital images on the display 628 multi-dimensional image/sequence 1010. Additionally, the user U may specify or select a loop number or repetition number 1304 of the digital image to set the loop number of images (n) of the plurality of 2D images 1000 of the scene S, wherein the images (n) of the plurality of 2D images 1000 of the scene S are sequentially displayed on the display 628, similar to FIG. 11 . Still further, the user U may specify or select a playback order 1306 of a digital image sequence or palindrome sequence for playback to set the display order of the images (n) of the multi-dimensional image/sequence 1010 of the scene S. The timed sequential display of the images produces appropriate binocular disparity through a motion pursuit ratio effect. It is contemplated that the computer system 10 and application 624 may utilize default or automatic settings herein.
DIFY，参照图14A和图14B，通过示例而非限制的方式，图示出了以设定的序列捕获的帧，这些帧以设定的序列回放给眼睛，以及人眼观看显示器628上的DIFY的感知的表示。解释DIFY及其几何形状以产生运动视差。运动视差是指一个点相对于一个静止点的角度变化。(运动追逐)。请注意，因为我们已经设置了关键主体KS点，所以前景的所有点都会向右移动，而背景的所有点都会向左移动。在图像反向的回文中，运动是相反的。在不同的视图中，任何点相对于关键主体的角度变化都会产生运动视差。DIFY, with reference to FIGS. 14A and 14B, illustrates, by way of example and not limitation, frames captured in a set sequence, played back to the eye in a set sequence, and a representation of the perception of the human eye viewing DIFY on display 628. DIFY and its geometry are interpreted to produce motion parallax. Motion parallax refers to the change in angle of a point relative to a stationary point. (Motion pursuit). Note that because we have set the key subject KS point, all points in the foreground will move to the right and all points in the background will move to the left. In a palindrome where the image is reversed, the motion is opposite. Any change in angle of any point relative to the key subject in different views will produce motion parallax.
DIFY是以设定的序列捕获的一系列帧，以设定的序列作为循环播放给眼睛。例如，图14A中描述了两个帧(假设是第一帧和最后一帧，如帧1101和1104)的回放情况。图14A表示物体的位置，如图4中的近平面NP物体在近平面NP上的位置及其与关键主体KS点在帧1101和1104中的关系，其中关键主体KS点由于施加在帧—帧1101、1102、1103和1104上的图像平移而恒定。图11A和图11B中的帧—帧1101、1102、1103和1104可以是重叠的，并且从主轴1112偏移了计算出的视差值(水平图像平移(HIT))，并且通过虚拟相机的间距进行预设。图14B通过示例而非限制的方式，图示出人眼从观看图14A的在显示器628上描绘的两帧(假定第一帧和最后一帧，例如帧1101和1104，其在近平面NP中的帧1为点1401，在近平面NP中的帧2为点1402)所感知到的情况，其中图像平面或屏幕平面与关键主体KS点和关键主体平面KSP相同，观看显示器628的用户U在显示器628前面或在显示器628与用户U眼睛—左眼LE和右眼RE之间观看虚拟深度近平面NP 1410。虚拟深度近平面NP 1410是近平面NP，因为它表示近平面NP中的帧1作为近平面点1401的物体，以及近平面NP中的帧2作为近平面点1402的物体，这是用户U眼—左眼LE和右眼RE在显示器628上观看多维图像序列1010时看到的最近的点。DIFY is a series of frames captured in a set sequence and played to the eye in a set sequence as a loop. For example, FIG. 14A describes the playback of two frames (assuming they are the first and last frames, such as frames 1101 and 1104). FIG. 14A shows the position of an object, such as the position of the near plane NP object on the near plane NP in FIG. 4 and its relationship with the key subject KS point in frames 1101 and 1104, where the key subject KS point is constant due to the image translation applied to the frames 1101, 1102, 1103 and 1104. The frames 1101, 1102, 1103 and 1104 in FIG. 11A and FIG. 11B can be overlapping and offset from the main axis 1112 by the calculated disparity value (horizontal image translation (HIT)), and are preset by the spacing of the virtual camera. FIG14B illustrates, by way of example and not limitation, what the human eye perceives from viewing two frames (assuming the first and last frames, e.g., frames 1101 and 1104, which are depicted on the display 628 of FIG14A as point 1401 for frame 1 and point 1402 for frame 2 in the near plane NP), wherein the image plane or screen plane is the same as the key subject point KS and the key subject plane KSP, and a user U viewing the display 628 is viewing a virtual depth near plane NP 1410 in front of the display 628 or between the display 628 and the user U's eyes—the left eye LE and the right eye RE. The virtual depth near plane NP 1410 is the near plane NP because it represents objects in frame 1 in the near plane NP as near plane point 1401, and objects in frame 2 in the near plane NP as near plane point 1402, which are the closest points seen by the user U's eyes—the left eye LE and the right eye RE—when viewing the multi-dimensional image sequence 1010 on the display 628.
虚拟深度近平面NP 1410将关键主体KS与近平面点1401中的物体和近平面点1402中的物体之间的视觉深度模拟为虚拟深度1420，即近平面NP与关键主体平面KSP之间的深度。这个深度是由于对同一个点(近平面点1401中的物体和近平面点1402中的物体)的两个视图之间的双眼视差产生的。近平面点1401中的物体和近平面点1402中的物体优选地是由于双眼视差而在时间上排序的不同的视图处的场景S中的相同点。此外，外射线1430和更具体的用户U眼睛—左眼LE和右眼RE的视角1440优选地与视网膜或眼轴成约二十七(27)度。(类似于利用显示器628的手机或平板电脑的景深)。这种描绘有助于定义场景S的组成的界限，近平面点1401和近平面点1402优选地位于景深内，外射线1430内，而近平面NP必须在外射线1430的内交叉位置1450之外。The virtual depth near plane NP 1410 simulates the visual depth between the key subject KS and the objects in the near plane point 1401 and the objects in the near plane point 1402 as a virtual depth 1420, i.e., the depth between the near plane NP and the key subject plane KSP. This depth is due to the binocular disparity between the two views of the same point (the object in the near plane point 1401 and the object in the near plane point 1402). The object in the near plane point 1401 and the object in the near plane point 1402 are preferably the same point in the scene S at different views that are temporally ordered due to binocular disparity. In addition, the external rays 1430 and more specifically the viewing angles 1440 of the user U's eyes - the left eye LE and the right eye RE, are preferably about twenty-seven (27) degrees from the retina or eye axis. (Similar to the depth of field of a mobile phone or tablet using a display 628). This depiction helps define the boundaries of the composition of the scene S, with near plane point 1401 and near plane point 1402 preferably located within the depth of field, within the outer ray 1430, while the near plane NP must be outside the inner intersection position 1450 of the outer ray 1430.
从X1到X2的运动是用户U眼睛—左眼LE和右眼RE将跟踪的运动。Xn是眼睛晶状体、左眼LE或右眼RE到虚拟近像平面1410上的图像点1411、1412的距离。X'n是从Xn的直角三角形到从眼球晶状体、左眼LE或右眼RE到虚拟近像平面1410上的图像点1411、1412到像平面628、KS、KSP的形成的路线(leg)的距离。平滑运动是在用户U眼睛—左眼LE和右眼RE观察到的点中的每个点处相对于关键主体KS的偏移引起的双眼视差。The movement from X1 to X2 is the movement that the user U eyes - left eye LE and right eye RE will track. Xn is the distance from the eye lens, left eye LE or right eye RE to the image points 1411, 1412 on the virtual near image plane 1410. X'n is the distance from the right triangle of Xn to the route (leg) formed from the eye lens, left eye LE or right eye RE to the image points 1411, 1412 on the virtual near image plane 1410 to the image plane 628, KS, KSP. The smooth motion is the binocular disparity caused by the offset relative to the key subject KS at each of the points observed by the user U eyes - left eye LE and right eye RE.
对于每只眼睛，左眼LE或右眼RE，可以相对于眼睛中心CL和眼内间隔中心—瞳孔间距离宽度IPD的一半—1440建立坐标系。两个角度β和α是用来解释DIFY运动追求的角度。β是当一条线从眼球晶状体—左眼LE和右眼RE—通过虚拟近平面1410到图像平面628、KS、KSP上的图像所形成的角度。Θ是β2-β1。而α是图像平面628、KS、KSP上两帧1101、1104的固定关键主体KS到虚拟近似图像平面1410上的点1411、1412的角度。α的变化代表眼睛的追逐。眼球旋转的运动跟随虚拟近似平面上的点的位置变化。而β是在左眼和右眼比较时负责平稳运动或双眼视差的角度。从眼球晶状体、左眼LE和右眼RE发出的连接到1440点的外射线1430代表景深或图像的边缘，即图像的一半。该线会随着虚拟相机景深的变化而变化。For each eye, left eye LE or right eye RE, a coordinate system can be established relative to the eye center CL and the center of the intraocular separation - half of the interpupillary distance width IPD - 1440. Two angles β and α are angles used to explain DIFY motion pursuit. β is the angle formed when a line is drawn from the eye lens - left eye LE and right eye RE - through the virtual near plane 1410 to the image on the image plane 628, KS, KSP. θ is β2-β1. And α is the angle from the fixed key subject KS of the two frames 1101, 1104 on the image plane 628, KS, KSP to the point 1411, 1412 on the virtual approximate image plane 1410. The change of α represents the pursuit of the eyes. The movement of the eye rotation follows the change of the position of the point on the virtual approximate plane. And β is the angle responsible for smooth motion or binocular disparity when comparing the left eye and the right eye. The outer ray 1430 from the eye lens, left eye LE and right eye RE, connected to the point 1440 represents the edge of the depth of field or the image, that is, half of the image. This line changes as the virtual camera's depth of field changes.
di/f＝Xidi/f＝Xi
如果我们把追逐运动定义为沿虚拟近平面的点的位置差，那么通过利用切线，我们可以得出：If we define pursuit motion as the position difference of points along the imaginary near plane, then by using tangents we can get:
这些方程向我们表明，追逐运动，X2-X1并不是视距的直接函数。随着视距的增加，感知到的深度di将变小，但由于角度差异小，运动将相对于图像的整个宽度保持大致相同。These equations show us that the pursuit motion, X2-X1, is not a direct function of viewing distance. As viewing distance increases, the perceived depth di will become smaller, but because the angular difference is small, the motion will remain roughly the same relative to the entire width of the image.
从数学上讲，视网膜运动与眼睛平滑追视率的比率决定了相对于人类视觉中心的固定点的深度。KSP的创建提供了创建深度所需的固定点。从数学上讲，所有的点都会以与任何其他点不同的方式移动，因为参考点在所有情况下都是相同的。Mathematically, the ratio of retinal motion to the eye's smooth pursuit rate determines the depth of a fixed point relative to the center of human vision. The creation of KSP provides the fixed point needed to create depth. Mathematically, all points will move differently than any other point because the reference point is the same in all cases.
现在参照图17，通过示例而非限制的方式，示出了与同视点(Horopter)弧或点和Panum区域融合的舒适圆(CoC)的代表性图示。同视点是空间中与注视点、同视点弧或点具有相同视差的点的轨迹。场景中落在同视点弧或点附近的对象是清晰的图像，而那些在同视点弧或点之外(在前面或后面)的对象是模糊的或模糊不清的。Panum是空间区域，Panum区域1720，围绕同视点以实现给定的眼部会聚度，内部界限为1721，外部界限为1722，其中投射到左眼和右眼LE/RE的不同点导致双眼融合，从而产生视觉深度的感觉，而位于该区域之外的点会导致复视——双重图像。此外，对于落在Panum区域内的对象，包括靠近同视点的对象，融合左眼和右眼的图像，用户U将看到单个清晰图像。在Panum的区域之外，无论是在前面还是在后面，用户U都会看到双重图像。Referring now to FIG. 17 , by way of example and not limitation, a representative illustration of a comfort circle (CoC) fused with a synoptophore (Horopter) arc or point and Panum's region is shown. A synoptophore is a locus of points in space that have the same disparity as the fixation point, a synoptophore arc or point. Objects in a scene that fall near the synoptophore arc or point are sharp images, while those outside (in front or behind) the synoptophore arc or point are blurred or indistinct. Panum is a region of space, the Panum region 1720, surrounding the synoptophore to achieve a given eye convergence, with inner limits 1721 and outer limits 1722, where different points projected to the left and right eye LE/RE result in binocular fusion, thereby producing a sense of visual depth, while points outside of this region result in diplopia - a double image. Furthermore, for objects that fall within the Panum region, including objects close to the synoptophore, the images of the left and right eyes are fused and the user U will see a single sharp image. Outside of the Panum's region, whether in front or behind, the user U will see a double image.
在此认识到，计算机系统10通过图像捕获应用624、图像操纵应用624、图像显示应用624可以利用不同的和单独定位的计算机系统10诸如一个或更多个用户系统220、222、224和应用程序206来执行。接下来，通过通信链路240和/或网络250、无线诸如5G，第二计算机系统10和应用程序206可以将场景S的图像集(n个)相对于关键主体平面传输，引入(左和右)双眼视差以在显示器628上显示多维数字图像，以使多个用户U在块或步骤1250中在显示器628上观看多维数字图像直播或作为重播/转播。It is recognized herein that the computer system 10, through the image capture application 624, the image manipulation application 624, the image display application 624, can be executed with different and separately located computer systems 10, such as one or more user systems 220, 222, 224 and the application 206. Next, through the communication link 240 and/or the network 250, wireless such as 5G, the second computer system 10 and the application 206 can transmit the image set (n) of the scene S relative to the key subject plane, introducing (left and right) binocular parallax to display the multi-dimensional digital image on the display 628, so that multiple users U can view the multi-dimensional digital image live or as a replay/rebroadcast on the display 628 in block or step 1250.
此外，图17示出了通过左和右像素1550L/R在显示器628上显示和观看多维图像1010，多维图像1010的光通过双凸透镜1540并且弯曲或折射以在显示器628上向左眼LE和右眼RE提供多维图像1010的3D观看，距像素1550的观看距离VD，其中近对象、关键主体KS和远对象在舒适圆CoC内，舒适圆CoC是近侧同视点弧或点并且在Panum区域1720内以实现在显示器628上的多维图像1010的与用户U的人类视觉系统舒适且兼容的清晰的单图像3D观看。In addition, Figure 17 shows the display and viewing of the multi-dimensional image 1010 on the display 628 through the left and right pixels 1550L/R, and the light of the multi-dimensional image 1010 passes through the double convex lens 1540 and is bent or refracted to provide 3D viewing of the multi-dimensional image 1010 to the left eye LE and the right eye RE on the display 628, at a viewing distance VD from the pixel 1550, where the near objects, key subjects KS and distant objects are within the comfort circle CoC, which is the near side isotropy arc or point and is within the Panum zone 1720 to achieve clear single-image 3D viewing of the multi-dimensional image 1010 on the display 628 that is comfortable and compatible with the human visual system of the user U.
区块链是一种共享分类账，该共享分类账促进网络中记录交易和追踪资产的过程。资产可以是有形的(房屋、汽车、现金、土地、艺术品或DIGY、3D立体及其数据集的所有权)或无形的(知识产权、专利、版权、品牌)。区块链是一种去中心化的、分布式的且通常公开的数字分类账，由称为区块的记录组成，这些记录用于使用密码学记录多个计算机上的交易，以便任何涉及的区块都不能被追溯地更改，而不需要更改所有后续区块。这允许参与者独立地验证和审计交易。每个区块包含数据或数据集、前一个区块(链)的加密哈希、时间戳和加密哈希(标识区块及其所有内容)。时间戳证明了，在块被发布以获得其哈希时交易数据已经存在。由于区块各自包含关于其前一个区块的信息，因此它们形成链，并且每个附加区块都会增强其之前的区块。因此，区块链难以对其数据进行修改，因为一旦记录，任何给定区块中的数据都不能在没有更改所有后续区块的情况下被追溯地更改。如果改变区块，则所有后续区块都无效并且具有不同的哈希。通过“工作证明”机制确保安全减慢了创建新的区块并进行分布式处理的交易速率。区块链通常由点对点网络(即一个开放且去中心化的数据库)管理，用作公开分布式分类账，其中节点共同遵守协议来通信并验证新的区块。每个新的区块都会发送给网络上的每个人，以验证新的区块，从而达成共识。虽然区块链记录并非是不可更改的，因为叉是可能的，但区块链可以经由分布式计算系统的设计被认为是安全的。A blockchain is a shared ledger that facilitates the process of recording transactions and tracking assets in a network. Assets can be tangible (ownership of a house, car, cash, land, artwork or DIY, 3D stereo and its data set) or intangible (intellectual property, patents, copyrights, brands). A blockchain is a decentralized, distributed and usually public digital ledger consisting of records called blocks that are used to record transactions on multiple computers using cryptography so that any block involved cannot be retroactively changed without changing all subsequent blocks. This allows participants to independently verify and audit transactions. Each block contains data or data sets, a cryptographic hash of the previous block (chain), a timestamp, and a cryptographic hash (identifying the block and all its contents). The timestamp proves that the transaction data existed at the time the block was published to obtain its hash. Since blocks each contain information about their previous block, they form a chain, and each additional block enhances the block before it. As a result, blockchains are difficult to modify their data because once recorded, the data in any given block cannot be retroactively changed without changing all subsequent blocks. If a block is changed, all subsequent blocks are invalid and have different hashes. Security is ensured by a "proof of work" mechanism that slows down the transaction rate at which new blocks are created and distributed processing. Blockchain is usually managed by a peer-to-peer network (i.e., an open and decentralized database) that serves as a public distributed ledger where nodes collectively follow a protocol to communicate and verify new blocks. Each new block is sent to everyone on the network to verify the new block and reach a consensus. Although blockchain records are not immutable because forks are possible, blockchain can be considered secure by design of a distributed computing system.
以太坊是一种流行的区块链标准以及其他私有和开源区块链算法。Ethereum is a popular blockchain standard along with other private and open source blockchain algorithms.
DIGY、3D立体、数据集和其他数据集以及验证创建者或作者的任何认证文档、真实性声明、历史、起源日期、所有权链、所有者链等(认证)可以包含在数据集(Dataset)中。数据集可以作为区块存储在区块链网络上，并且此后与数据集相关的每个后续交易可以在后续区块中阐述，每个后续区块包含前一个区块的加密哈希、数据集交易数据去中心化信任源的时间戳。DIGY, 3D, datasets and other datasets, as well as any certification documents verifying the creator or author, authenticity statements, history, origin date, ownership chain, owner chain, etc. (certification) can be included in the dataset. The dataset can be stored as a block on the blockchain network, and each subsequent transaction related to the dataset can be set forth in a subsequent block, each subsequent block containing a cryptographic hash of the previous block and a timestamp of the decentralized trusted source of the dataset transaction data.
利用不可替代令牌——对于选定的DIGY、DIGY序列或立体3D图像文件、数据集或其他数据集，不可替代令牌(NFT)是存储在区块链上的独特可收藏加密资产，具有区分DIGY、DIGY序列或立体3D图像文件、数据集或来自任何其他数字内容的数据集的唯一的识别码和元数据，使其成为独一无二的一种或限量版的数字内容。能够创建独特的数字内容和/或一定的副本数——独一无二的或有限的无法复制的数字纪念品数的真品证书。纪念品存储在区块链网络上。它是“不可替代的”，因为它不能像通过互联网自由移动的类似可复制内容一样容易地进行交换。此外，利用DIGY、DIGY序列或立体3D图像文件、艺术品数据集及评估以及任何认证文件来对珍贵的艺术品进行验证证明了创作者或作者，真实性声明、历史、起源日期、所有权链、所有者链等(身份验证)可以包括在数据集(Dataset)中，并且制作智能合约或数据集的NFT。Utilize Non-Fungible Tokens - For selected DIGY, DIGY Sequence or Stereoscopic 3D Image Files, Datasets or other Datasets, Non-Fungible Tokens (NFTs) are unique collectible cryptographic assets stored on the blockchain with unique identifiers and metadata that distinguish DIGY, DIGY Sequence or Stereoscopic 3D Image Files, Datasets or Datasets from any other digital content, making it a one-of-a-kind or limited edition digital content. Enables the creation of unique digital content and/or a certain number of copies - a certificate of authenticity for a unique or limited number of digital memorabilia that cannot be reproduced. The memorabilia is stored on the blockchain network. It is "non-fungible" because it cannot be exchanged as easily as similar reproducible content that moves freely over the Internet. In addition, the use of DIGY, DIGY Sequence or Stereoscopic 3D Image Files, Artwork Datasets and Assessments and any certification documents to verify the creator or author of the precious artwork, authenticity statements, history, origin date, chain of ownership, owner chain, etc. (Authentication) can be included in the Dataset (Dataset) and make a smart contract or NFT of the Dataset.
NFT只是记录谁拥有独特的数字内容。该内容可以是艺术、音乐、照片、3D图形、推文、备忘录、游戏、视频、GIF等任何内容。只要它是数字内容，并且只要它被创建，它都可以成为NFT。NFTs simply record who owns unique digital content. That content can be anything from art, music, photos, 3D graphics, tweets, memos, games, videos, GIFs, etc. As long as it is digital content, and as long as it is created, it can become an NFT.
当前的NFT数字内容可以以盗版由第三方装置捕获并经由互联网大量散布，并且使标准的NFT内容的NFT数字内容贬值。Current NFT digital content can be captured by third-party devices in a pirated form and distributed in large quantities via the Internet, devaluing the NFT digital content of standard NFT content.
此处的数字内容是由艺术家用户生成的DIGY和3D立体图像，并使用我们上面的DIGY和3D立体平台基于DIGY和3D立体图像创建自己的NFT艺术品。The digital content here is DIGY and 3D stereo images generated by artist users, and use our DIGY and 3D stereo platform above to create their own NFT artwork based on DIGY and 3D stereo images.
现在参照图18，示出了创建用于DIGY、DIGY序列、立体3D图像文件或其他数据文件(Dataset)的NFT的方法的流程图1800。Referring now to FIG. 18 , there is shown a flowchart 1800 of a method for creating an NFT for a DIGY, DIGY sequence, stereoscopic 3D image file, or other data file (Dataset).
在块或步骤1810中，打开、点击或启动由图13和图16B表示的智能装置(计算机系统10)上的应用程序NFT(应用程序206)。In block or step 1810, the application NFT (application 206) on the smart device (computer system 10) represented by Figures 13 and 16B is opened, clicked or launched.
在块或步骤1815中，点击/触摸/选择智能装置上的“创建NFT”。In block or step 1815, click/touch/select “Create NFT” on the smart device.
在块或步骤1820中，点击/触摸/选择/访问显示器628上的“DIGY/DIGY序列/立体3D图像文件库”图标以访问智能装置上的图像文件等。In block or step 1820 , click/touch/select/access the “DIGY/DIGY Sequence/Stereoscopic 3D Image File Library” icon on the display 628 to access image files, etc. on the smart device.
在块或步骤1825中，点击/触摸/选择来自智能装置10的库主存储装置214的DIGY、DIGY序列、立体3D图像文件或其他数据文件(Dataset)文件。In block or step 1825 , click/touch/select a DIGY, DIGY sequence, stereoscopic 3D image file or other data file (Dataset) file from the library main storage device 214 of the smart device 10 .
在块或步骤1830中，点击/触摸/选择智能装置10的显示器628上的“创建NFT”。In block or step 1830 , click/touch/select “Create NFT” on the display 628 of the smart device 10 .
在块或步骤1835中，为选择的图像文件——DIGY、DIGY序列、立体3D图像文件或其他数据文件(Dataset)——创建NFT。智能装置10的用户经由系统或应用程序将利用区块链提供商中的一者为其DIGY、DIGY序列、立体3D图像文件或其他数据文件(Dataset)发行NFT。以太坊是目前领先的用于NFT发行的区块链服务。然而，还有一系列其他区块链正变得越来越流行。其中的一者可以由智能装置制造商或其他服务提供商提供。In block or step 1835, an NFT is created for the selected image file - DIGY, DIGY sequence, stereoscopic 3D image file or other data file (Dataset). The user of the smart device 10 will use one of the blockchain providers to issue NFTs for its DIGY, DIGY sequence, stereoscopic 3D image file or other data file (Dataset) via the system or application. Ethereum is currently the leading blockchain service for NFT issuance. However, there are a series of other blockchains that are becoming more and more popular. One of them can be provided by the smart device manufacturer or other service provider.
在块或步骤1840中，利用NFT令牌标准、兼容的钱包服务和市场来提供DIGY、DIGY序列、立体3D图像文件或其他数据文件(Dataset)的NFT销售或交换。一旦创建，用户就可以提供其NFT，诸如NFT-DIGY、DIGY序列或立体3D图像文件，用于在与NFT兼容的钱包服务和市场(诸如OpenSea，其是基于以太坊的NFT市场)上销售或验证。类似的市场可以由智能装置制造商或其他服务提供商提供。In block or step 1840, NFT token standards, compatible wallet services and markets are used to provide NFT sales or exchanges of DIGY, DIGY sequences, stereoscopic 3D image files, or other data files (Dataset). Once created, users can offer their NFTs, such as NFT-DIGY, DIGY sequences, or stereoscopic 3D image files, for sale or verification on NFT-compatible wallet services and markets (such as OpenSea, which is an Ethereum-based NFT market). Similar markets can be provided by smart device manufacturers or other service providers.
DIGY、DIGY序列、立体3D图像文件或其他数据文件(Dataset)的独特之处在于，它们不能被第三方装置捕获，该第三方装置诸如为查看DIGY、DIGY序列或立体3D的的其他智能装置，因为此类装置无法访问原始数字文件并且可能没有DIGY和3D立体平台的许可证。当前的NFT数字内容可以以盗版由第三方装置捕获并经由互联网大量散布，并且使标准的NFT内容的NFT数字内容贬值。DIGY, DIGY sequence, stereoscopic 3D image files or other data files (Dataset) are unique in that they cannot be captured by third-party devices, such as other smart devices for viewing DIGY, DIGY sequence or stereoscopic 3D, because such devices cannot access the original digital files and may not have licenses for DIGY and 3D stereo platforms. Current NFT digital content can be captured by third-party devices in pirated form and distributed in large quantities via the Internet, devaluing the NFT digital content of standard NFT content.
此处的DIGY可以包括2D视频、2D图像拼贴、3DDIGY。DIGY here can include 2D video, 2D image collage, and 3D DIGY.
序列是将多个DIGYS串联在一起或按顺序循环以创建一个故事。A sequence is multiple DIGYS strung together or looped in sequence to create a story.
现在参照图19，示出了将多个DIGY–MP4文件按顺序或作为循环链接或循环的方法的流程图1900，其中有或没有音频文件，诸如AAC(m4a)格式、AIFF、Apple Lossless、MP3和WAV或其他相似的存储在那里的音频格式。Referring now to FIG. 19 , there is shown a flowchart 1900 of a method of linking or looping multiple DIGY-MP4 files in sequence or as a loop, with or without audio files such as AAC (m4a) format, AIFF, Apple Lossless, MP3 and WAV or other similar audio formats stored therein.
在块或步骤1910中，打开、点击或启动由图13和图16B表示的智能装置(计算机系统10)的显示器628上的应用程序Photon3D(应用程序206)。In block or step 1910, the application Photon3D (application 206) on the display 628 of the smart device (computer system 10) represented by Figures 13 and 16B is opened, clicked or launched.
在块或步骤1915中，点击/触摸/选择智能装置10的显示器上的“创建DIGY序列”。In block or step 1915 , click/touch/select “Create DIGY Sequence” on the display of the smart device 10 .
在块或步骤1920中，点击/触摸/选择显示器628上的“DIGY库”图标以访问例如智能装置10上的DIGY MP4文件等。In block or step 1920 , click/touch/select the “DIGY Library” icon on the display 628 to access, for example, DIGY MP4 files on the smart device 10 , etc.
在块或步骤1925中，将来自智能装置10的DIGY库主存储装置214中的第一DIGY文件点击/触摸/拖动/选择到时间线。In block or step 1925, click/touch/drag/select the first DIGY file from the DIGY library primary storage device 214 of the smart device 10 to the timeline.
在块或步骤1930中，点击/触摸/拖动/选择来自智能装置10的显示器上的DIGY库中的第二或多个DIGY文件并将它们按顺序或期望的顺序放置。In block or step 1930 , click/touch/drag/select a second or more DIGY files from the DIGY library on the display of the smart device 10 and place them in a sequential or desired order.
在块或步骤1935中，修剪/编辑/裁剪DIGY文件或设置持续时间(开始/停止/速度)。通过设置过渡/剪切来布置DIGY。添加可选的音频文件/音效作为第二个音轨并混合声音幅度。In block or step 1935, trim/edit/crop the DIGY file or set duration (start/stop/speed). Lay out the DIGY by setting transitions/cuts. Add optional audio file/sound effects as a second track and mix the sound amplitude.
在块或步骤1940中，通过经由将DIGY文件拖到不同的顺序或位置进行点击/触摸/选择/拖动来对DIGY文件进行重新排序是顺序。In block or step 1940, the DIGY files are reordered by clicking/touching/selecting/dragging to drag the DIGY files to a different order or position.
在块或步骤1945中，通过点击/触摸/选择保存来保存完成的DIGY序列–MP4，以将完成的图像和音频MP4文件保存在智能装置10的主存储装置214中。In block or step 1945 , save the completed DIGY Sequence – MP4 by clicking/touching/selecting Save to save the completed image and audio MP4 files in the primary storage device 214 of the smart device 10 .
在块或步骤1950中，在智能装置(计算机系统10)上查看和收听完成的DIGY序列–MP4文件。In block or step 1950, the completed DIGY sequence - MP4 file is viewed and listened to on the smart device (computer system 10).
在块或步骤1955中，可以将共享DIGY序列–MP4文件附加到电子邮件或文本、空投或上传到社交媒体进行共享。In block or step 1955, the shared DIGY sequence – MP4 file can be attached to an email or text, airdropped, or uploaded to social media for sharing.
本文预期，用户可以匹配两个不同DIGY之间的过渡，无论是使用音频峰值检测还是甚至时间特征/节奏匹配算法BPM以手动方式或以编程方式实现：每分钟120拍；拍号：4/4.4小节＝4节段(DIGY)，具有3个过渡。[DIGY 1-DIGY 2-DIGY 3-DIGY4]。4小节x每小节4节拍＝16节拍；16节拍@120bpm＝8秒；4节段(DIGY)@每节段2秒＝8秒。每节段1个DIGY＝4个DIGY。It is contemplated herein that a user can match transitions between two different DIGYs, whether manually or programmatically using audio peak detection or even temporal signature/rhythm matching algorithms BPM: 120 beats per minute; Time signature: 4/4.4 bars = 4 segments (DIGYs) with 3 transitions. [DIGY 1-DIGY 2-DIGY 3-DIGY4]. 4 bars x 4 beats per bar = 16 beats; 16 beats @ 120bpm = 8 seconds; 4 segments (DIGYs) @ 2 seconds per segment = 8 seconds. 1 DIGY per segment = 4 DIGYs.
本文预期，用户可以使用瞬态检测来找到小节开始处的下拍，并且然后将其与第一DIGY的开始同步。每个DIGY被修剪至2秒，然后一起排序以匹配排序后的DIGY的音乐过渡。It is contemplated herein that the user can use transient detection to find the downbeat at the start of a measure and then synchronize it with the start of the first DIGY. Each DIGY is trimmed to 2 seconds and then sequenced together to match the musical transitions of the sequenced DIGYs.
现在参照图20，示出了操纵DIGY或DIGY序列图像文件以与音频文件同步的方法的流程图2000。Referring now to FIG. 20 , there is shown a flow chart 2000 of a method of manipulating a DIGY or DIGY sequence image file to synchronize with an audio file.
在块或步骤2010中，打开、点击或启动由图13和图16B表示的智能装置(计算机系统10)的显示器628上的应用程序Photon3D(应用程序206)。In block or step 2010, the application Photon3D (application 206) on the display 628 of the smart device (computer system 10) represented by Figures 13 and 16B is opened, clicked or launched.
在块或步骤2015中，经由对智能装置10的显示器628的输入点击/触摸/选择“音乐应用程序”。In block or step 2015 , “Music App” is clicked/touched/selected via input to the display 628 of the smart device 10 .
在块或步骤2020中，经由对智能装置10的显示器628的输入点击/触摸/选择DIGY、DIGY序列、立体3D图像文件(数据集)，以经由来自所述显示器628的输入从存储装置604、606选择数据集文件。In block or step 2020 , click/touch/select DIGY, DIGY sequence, stereoscopic 3D image file (dataset) via input to the display 628 of the smart device 10 to select a dataset file from the storage device 604 , 606 via input from the display 628 .
在块或步骤2025中，经由处理器102执行指令206以准备，经由来自所述显示器628的输入将DIGY.gif文件转换成DIGY.mps文件；经由来自显示器628的输入创建DIGY帧时间线；经由来自所述显示器206的输入将图像帧(3DDIGY、2D图像、2D视频)导入到所述帧时间线中；经由来自所述显示器206的输入来调整序列/帧停留时间。In block or step 2025, instructions 206 are executed via processor 102 to prepare for converting a DIGY.gif file into a DIGY.mps file via input from the display 628; creating a DIGY frame timeline via input from the display 628; importing image frames (3DDIGY, 2D images, 2D video) into the frame timeline via input from the display 206; and adjusting sequence/frame dwell time via input from the display 206.
在块或步骤2030中，经由处理器102执行指令206以经由智能装置10中的麦克风记录音频文件，经由来自所述显示器628的输入从存储装置604、606或诸如ITUNES的在线服务选择音频文件，并且经由来自所述显示器628的输入将音频文件导入或下载到时间线。本文预期，DIGY序列和音频文件可以被转换为.mp4文件以便经由网络250与其他智能装置222共享。In block or step 2030, instructions 206 are executed via the processor 102 to record an audio file via a microphone in the smart device 10, select an audio file from the storage device 604, 606 or an online service such as ITUNES via input from the display 628, and import or download the audio file to the timeline via input from the display 628. It is contemplated herein that the DIGY sequence and audio files may be converted to .mp4 files for sharing with other smart devices 222 via the network 250.
在块或步骤2035中，经由处理器102执行指令206以经由来自显示器628的输入将音频文件从存储器装置604、606拖动、叠加或放置到DIGY帧时间线。In block or step 2035 , instructions 206 are executed via processor 102 to drag, overlay or drop the audio file from the memory device 604 , 606 to the DIGY frame timeline via input from the display 628 .
在块或步骤2040中，经由处理器102执行指令206以经由来自显示器628的输入相对于DIGY图像文件调整、裁剪、链接或布置音频文件。In block or step 2040 , instructions 206 are executed via processor 102 to adjust, crop, link, or arrange the audio file relative to the DIGY image file via input from display 628 .
在块或步骤2045中，通过点击/触摸/选择保存来保存完成的图像和音频文件–MP4，以经由来自显示器628的输入将完成的图像和音频MP4文件保存在主存储装置214中。In block or step 2045 , Save Completed Image and Audio File—MP4 by clicking/touching/selecting Save to save the completed image and audio MP4 file in the primary storage device 214 via input from the display 628 .
在块或步骤2050中，经由来自显示器628的输入在智能装置(计算机系统10)上播放、查看和收听完成的图像和音频文件。In block or step 2050 , the completed image and audio files are played, viewed, and heard on the smart device (computer system 10 ) via input from the display 628 .
在框或步骤2055中，可以将共享DIGY和音频(文件)附加到电子邮件或文本、空投，或者上传到社交媒体以共享。In box or step 2055, the shared DIGY and audio (file) can be attached to an email or text, airdropped, or uploaded to social media for sharing.
关于上述描述，应该认识到，最佳的尺寸关系，包括大小、材料、形状、形式、位置、移动机构、功能和操作方式、装配和使用的变化，都旨在是本公开所要涵盖的。With respect to the above description, it should be recognized that optimal dimensional relationships, including variations in size, material, shape, form, position, movement mechanisms, functions and modes of operation, assembly and use are intended to be covered by the present disclosure.
前述描述和附图包括例例示性实施方式。在这样描述了示例性实施方式之后，本领域的技术人员应该注意到，本公开仅是示例性的，并且可以在本公开的范围内进行各种其他替代、调整和修改。仅仅按照一定的顺序列出或编号方法的步骤并不构成对该方法的步骤顺序的任何限制。本公开所属领域的技术人员在受益于前述描述和相关附图中所呈现的教导后，将想到许多修改和其他实施方式。尽管此处可以采用特定术语，但它们仅在一般和描述性意义上使用，而不是出于限制的目的。此外，已经详细描述了本公开，应当理解，在不脱离所附权利要求所限定的本公开的精神和范围的情况下，可以对其进行各种更改、替换和改变。因此，本公开不限于本文所示的具体实施方式，而是仅受所附权利要求的限制。The foregoing description and the accompanying drawings include examples of illustrative embodiments. Having thus described the exemplary embodiments, it should be noted by those skilled in the art that the present disclosure is merely exemplary and that various other substitutions, adjustments and modifications may be made within the scope of the present disclosure. Simply listing or numbering the steps of a method in a certain order does not constitute any limitation on the order of the steps of the method. Having benefited from the teachings presented in the foregoing description and the associated drawings, those skilled in the art will be able to think of many modifications and other embodiments. Although specific terms may be employed herein, they are used only in a general and descriptive sense, and not for the purpose of limitation. In addition, having described the present disclosure in detail, it should be understood that various changes, substitutions and modifications may be made thereto without departing from the spirit and scope of the present disclosure as defined by the appended claims. Therefore, the present disclosure is not limited to the specific embodiments shown herein, but is only limited by the appended claims.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US202163197941P | 2021-06-07 | 2021-06-07 | |
| US63/197,941 | 2021-06-07 | ||
| US202163212025P | 2021-06-17 | 2021-06-17 | |
| US63/212,025 | 2021-06-17 | ||
| US17/355,906 | 2021-06-23 | ||
| US17/355,906US20210321077A1 (en) | 2020-01-09 | 2021-06-23 | 2d digital image capture system and simulating 3d digital image sequence |
| US17/459,067US20210392314A1 (en) | 2020-01-09 | 2021-08-27 | Vehicle terrain capture system and display of 3d digital image and 3d sequence |
| US17/459,067 | 2021-08-27 | ||
| US17/511,490US12106501B2 (en) | 2020-01-09 | 2021-10-26 | Subsurface imaging and display of 3D digital image and 3D image sequence |
| US17/511,490 | 2021-10-26 | ||
| US17/525,246 | 2021-11-12 | ||
| US17/525,246US12108015B2 (en) | 2020-01-09 | 2021-11-12 | 2D digital image capture system, frame speed, and simulating 3D digital image sequence |
| US29/816,462 | 2021-11-22 | ||
| US29/816,461 | 2021-11-22 | ||
| US29816462 | 2021-11-22 | ||
| US29816461 | 2021-11-22 | ||
| PCT/US2022/032524WO2022261111A1 (en) | 2021-06-07 | 2022-06-07 | 2d digital image capture system and simulating 3d digital image and sequence |
| US17/834,023US12256141B2 (en) | 2021-05-28 | 2022-06-07 | 2D digital image capture system and simulating 3D digital image and sequence |
| Publication Number | Publication Date |
|---|---|
| CN117897951Atrue CN117897951A (en) | 2024-04-16 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202280047753.0APendingCN117897951A (en) | 2021-06-07 | 2022-06-07 | 2D digital image capture system and simulation of 3D digital images and sequences |
| Country | Link |
|---|---|
| CN (1) | CN117897951A (en) |
| WO (2) | WO2022261105A1 (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO1993021559A1 (en)* | 1992-04-17 | 1993-10-28 | Noritsu Koki Co., Ltd. | Printing method and apparatus of photograph |
| US20060023197A1 (en)* | 2004-07-27 | 2006-02-02 | Joel Andrew H | Method and system for automated production of autostereoscopic and animated prints and transparencies from digital and non-digital media |
| US10848731B2 (en)* | 2012-02-24 | 2020-11-24 | Matterport, Inc. | Capturing and aligning panoramic image and depth data |
| US10178247B2 (en)* | 2015-01-30 | 2019-01-08 | Jerry Nims | Digital multi-dimensional image photon platform system and methods of use |
| US9992473B2 (en)* | 2015-01-30 | 2018-06-05 | Jerry Nims | Digital multi-dimensional image photon platform system and methods of use |
| US10074012B2 (en)* | 2016-06-17 | 2018-09-11 | Dolby Laboratories Licensing Corporation | Sound and video object tracking |
| JP6829015B2 (en)* | 2016-06-23 | 2021-02-10 | マクセル株式会社 | Mobile terminal |
| US10917633B2 (en)* | 2017-09-15 | 2021-02-09 | Intel Corporation | Generating stereoscopic light field panoramas using concentric viewing circles |
| CN109756723B (en)* | 2018-12-14 | 2021-06-11 | 深圳前海达闼云端智能科技有限公司 | Method and apparatus for acquiring image, storage medium and electronic device |
| EP3694208A1 (en)* | 2019-02-05 | 2020-08-12 | Jerry Nims | A method and system for simulating a 3-dimensional image sequence |
| US12108015B2 (en)* | 2020-01-09 | 2024-10-01 | JUC Holdings Limited | 2D digital image capture system, frame speed, and simulating 3D digital image sequence |
| Publication number | Publication date |
|---|---|
| WO2022261111A1 (en) | 2022-12-15 |
| WO2022261105A1 (en) | 2022-12-15 |
| Publication | Publication Date | Title |
|---|---|---|
| US12126896B2 (en) | 2D digital image capture system and simulating 3D digital image and sequence | |
| Kwon et al. | A study of layout, rendering, and interaction methods for immersive graph visualization | |
| Balram et al. | Light‐field imaging and display systems | |
| US20210392314A1 (en) | Vehicle terrain capture system and display of 3d digital image and 3d sequence | |
| US12108015B2 (en) | 2D digital image capture system, frame speed, and simulating 3D digital image sequence | |
| US12341945B2 (en) | 2D image capture system and display of 3D digital image | |
| Gao et al. | Correcting geometric distortions in stereoscopic 3D imaging | |
| US20250233970A1 (en) | Single 2d image capture system, processing & display of 3d digital image | |
| US20220232201A1 (en) | Image generation system and method | |
| Dąbała et al. | Manipulating refractive and reflective binocular disparity | |
| US20210297647A1 (en) | 2d image capture system, transmission & display of 3d digital image | |
| US20210321077A1 (en) | 2d digital image capture system and simulating 3d digital image sequence | |
| CN116097644A (en) | 2D digital image capturing system and analog 3D digital image sequence | |
| CN117897951A (en) | 2D digital image capture system and simulation of 3D digital images and sequences | |
| CN118541968A (en) | 2D digital image capturing system and analog 3D digital image and sequence | |
| EP4352953A1 (en) | 2d digital image capture system and simulating 3d digital image and sequence | |
| WO2022093376A1 (en) | Vehicle terrain capture system and display of 3d digital image and 3d sequence | |
| US20250139866A1 (en) | Single 2d digital image capture system processing, displaying of 3d digital image sequence | |
| US20250233971A1 (en) | Single 2d image capture system, processing & display of 3d digital image | |
| Regalbuto | Remote Visual Observation of Real Places through Virtual Reality Headsets | |
| US20250055967A1 (en) | Single 2d digital image capture system processing, displaying of 3d digital image sequence | |
| WO2021247405A1 (en) | 2d image capture system & display of 3d digital image | |
| WO2021247416A1 (en) | 2d image capture system, transmission & display of 3d digital image | |
| CN117203668A (en) | 2D digital image capturing system, frame rate and analog 3D digital image sequence | |
| WO2022104077A1 (en) | 2d digital image capture system, frame speed, and simulating 3d digital image sequence |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |