US20150253974A1 - Control of large screen display using wireless portable computer interfacing with display controller - Google Patents

Abstract

A multi-window user interface (UI) on a control device such as a tablet computer communicates commands to a display controller, which may be implemented by a game console. The controller controls presentation on a large screen display according to the commands. The control device may always present a multi-window UI which the control device can command to have mimicked on the display, with the UI also being configured to cause a single video to be presented whole screen on the display while the multi-window UI remains presented on the control device. The video feeds that populate the multi-window UI are of the same content but of lower resolution than those which are presented on the display. A movable window on the control device can be dragged and dropped to cause a corresponding magnifying focus on the display over the corresponding portion of video in focus on the control device.

Description

I. FIELD OF THE INVENTION
• The application relates generally to controlling a large screen display using a wireless portable computer such as a tablet or laptop computer interfacing with a display controller such as a game console.
II. BACKGROUND OF THE INVENTION
• A computer ecosystem, or digital ecosystem, is an adaptive and distributed socio-technical system that is characterized by its sustainability, self-organization, and scalability. Inspired by environmental ecosystems, which consist of biotic and abiotic components that interact through nutrient cycles and energy flows, complete computer ecosystems consist of hardware, software, and services that in some cases may be provided by one company, such as Sony. The goal of each computer ecosystem is to provide consumers with everything that may be desired, at least in part services and/or software that may be exchanged via the Internet. Moreover, interconnectedness and sharing among elements of an ecosystem, such as applications within a computing cloud, provides consumers with increased capability to organize and access data and presents itself as the future characteristic of efficient integrative ecosystems.
• Two general types of computer ecosystems exist: vertical and horizontal computer ecosystems. In the vertical approach, virtually all aspects of the ecosystem are owned and controlled by one company, and are specifically designed to seamlessly interact with one another. Horizontal ecosystems, one the other hand, integrate aspects such as hardware and software that are created by other entities into one unified ecosystem. The horizontal approach allows for greater variety of input from consumers and manufactures, increasing the capacity for novel innovations and adaptations to changing demands.
SUMMARY OF THE INVENTION
• An example ecosystem that is pertinent here is an entertainment ecosystem in the home or in a luxury suite at a stadium that includes a large screen high definition display controlled by a controller such as a personal computer (PC) or game console which receives commands from a portable control device such as a tablet computer.
• Accordingly, a control device includes at least one computer readable storage medium bearing instructions executable by a processor, and at least one processor configured for accessing the computer readable storage medium to execute the instructions to configure the processor for presenting on a display of the control device a user interface (UI) having plural selectors arranged in a layout. Each selector is established by a respective video feed. The instructions when executed by the processor configure the processor for receiving a first user input corresponding to a first command, and wirelessly transmitting the first command to a controller of the display device to command the controller to present plural video feeds on the display device. The video feeds on the control device are of the same content as respective video feeds on the display device. The instructions when executed by the processor configure the processor for receiving a second user input corresponding to a second command, and for wirelessly transmitting the second command to the controller to command the controller to present a single video feed full screen on the display device with the layout of the selectors on the control device remaining unchanged.
• In some embodiments, the first user input is established by a user touch on any portion of a display of the control device, and the second user input can be established only by a user touch on a selector corresponding to the single video feed.
• If desired, the instructions when executed by the processor may configure the processor for transmitting the first command to the controller of the display device to command the controller to present the plural video feeds on the display device in a same layout as the video feeds establishing the selectors are arranged on the display of the monitoring device. In some examples, the instructions when executed by the processor configure the processor for receiving a third user input corresponding to a third command. The third user input can be a drag of an image associated with a content selector and a drop of the image onto a portion of the control device for commanding the controller to cause the display device to present a video corresponding to the content selector in a portion of the display device that corresponds to the portion of the control device onto which the image was dropped.
• The control device can be established by a portable computer.
• The instructions when executed by the processor may if desired configure the processor for providing a visible indicator on the control device that the display device has been commanded to present the single video feed full screen on the display device.
• In another aspect, a system includes at least one display device configured for presenting plural video contents in respective windows of the display device, at least one controller configured for controlling the display device, and at least one control device configured for communicating commands to the controller to control presentation on the display device. The control device is configured to present a user interface (UI) having plural selectors arranged in a layout on a screen of the control device. Each selector is established by a respective video feed having a first resolution. The control device is configured for causing the display device to present plural video feeds on the display device, with the video feeds on the control device being of the same content as respective video feeds on the display device. The respective video feeds on the display device have second resolution higher than the first resolution. Furthermore, the control device is configured for causing a single video feed full screen to be presented on the display device with the layout of the selectors on the control device remaining unchanged.
• In another aspect, a method includes sending plural video feeds having a first resolution to a control device over a computer network. The method also includes sending plural video feeds having a second resolution to a display device over the computer network, with the second resolution being higher than the first. The video feeds of the first resolution are the same content as the video feeds of the second content. Video presentation on the display device is controlled at least in part based on user touches of the video feeds presented on the control device.
• The details of the present invention, both as to its structure and operation, can be best understood in reference to the accompanying drawings, in which like reference numerals refer to like parts, and in which:
BRIEF DESCRIPTION OF THE DRAWINGS
• FIG. 1 is a block diagram of an example system including an example in accordance with present principles;
• FIG. 2 is a partially schematic view of a specific example system with two UHD displays mounted on a wall side by side;
• FIG. 3 is a schematic diagram illustrating control of video presentation on a display using a movable window on a control device; and
• FIGS. 4-7 are each a series of screen shots illustrating various aspects of example embodiments.
DETAILED DESCRIPTION
• This disclosure relates generally to computer ecosystems including aspects of consumer electronics (CE) device based user information in computer ecosystems. A system herein may include server and client components, connected over a network such that data may be exchanged between the client and server components. The client components may include one or more computing devices including portable televisions (e.g. smart TVs, Internet-enabled TVs), portable computers such as laptops and tablet computers, and other mobile devices including smart phones and additional examples discussed below. These client devices may operate with a variety of operating environments. For example, some of the client computers may employ, as examples, operating systems from Microsoft, or a Unix operating system, or operating systems produced by Apple Computer or Google. These operating environments may be used to execute one or more browsing programs, such as a browser made by Microsoft or Google or Mozilla or other browser program that can access web applications hosted by the Internet servers discussed below.
• Servers may include one or more processors executing instructions that configure the servers to receive and transmit data over a network such as the Internet. Or, a client and server can be connected over a local intranet or a virtual private network. A server or controller may be instantiated by a game console such as a Sony Playstation (trademarked), a personal computer, etc.
• Information may be exchanged over a network between the clients and servers. To this end and for security, servers and/or clients can include firewalls, load balancers, temporary storages, and proxies, and other network infrastructure for reliability and security. One or more servers may form an apparatus that implement methods of providing a secure community such as an online social website to network members.
• As used herein, instructions refer to computer-implemented steps for processing information in the system. Instructions can be implemented in software, firmware or hardware and include any type of programmed step undertaken by components of the system.
• A processor may be any conventional general purpose single- or multi-chip processor that can execute logic by means of various lines such as address lines, data lines, and control lines and registers and shift registers.
• Software modules described by way of the flow charts and user interfaces herein can include various sub-routines, procedures, etc. Without limiting the disclosure, logic stated to be executed by a particular module can be redistributed to other software modules and/or combined together in a single module and/or made available in a shareable library.
• Present principles described herein can be implemented as hardware, software, firmware, or combinations thereof; hence, illustrative components, blocks, modules, circuits, and steps are set forth in terms of their functionality.
• Further to what has been alluded to above, logical blocks, modules, and circuits described below can be implemented or performed with a general purpose processor, a digital signal processor (DSP), a field programmable gate array (FPGA) or other programmable logic device such as an application specific integrated circuit (ASIC), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A processor can be implemented by a controller or state machine or a combination of computing devices.
• The functions and methods described below, when implemented in software, can be written in an appropriate language such as but not limited to C# or C++, and can be stored on or transmitted through a computer-readable storage medium such as a random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), compact disk read-only memory (CD-ROM) or other optical disk storage such as digital versatile disc (DVD), magnetic disk storage or other magnetic storage devices including removable thumb drives, etc. A connection may establish a computer-readable medium. Such connections can include, as examples, hard-wired cables including fiber optics and coaxial wires and digital subscriber line (DSL) and twisted pair wires. Such connections may include wireless communication connections including infrared and radio.
• Components included in one embodiment can be used in other embodiments in any appropriate combination. For example, any of the various components described herein and/or depicted in the Figures may be combined, interchanged or excluded from other embodiments.
• “A system having at least one of A, B, and C” (likewise “a system having at least one of A, B, or C” and “a system having at least one of A, B, C”) includes systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc.
• Now specifically referring toFIG. 1, an example ecosystem10 is shown, which may include one or more of the example devices mentioned above and described further below in accordance with present principles. The first of the example devices included in the system10 is an example primary display device, and in the embodiment shown is an audio video display device (AVDD)12 such as but not limited to an Internet-enabled TV. Thus, theAVDD12 alternatively may be an appliance or household item, e.g. computerized Internet enabled refrigerator, washer, or dryer. TheAVDD12 alternatively may also be a computerized Internet enabled (“smart”) telephone, a tablet computer, a notebook computer, a wearable computerized device such as e.g. computerized Internet-enabled watch, a computerized Internet-enabled bracelet, other computerized Internet-enabled devices, a computerized Internet-enabled music player, computerized Internet-enabled head phones, a computerized Internet-enabled implantable device such as an implantable skin device, etc. Regardless, it is to be understood that theAVDD12 is configured to undertake present principles (e.g. communicate with other CE devices to undertake present principles, execute the logic described herein, and perform any other functions and/or operations described herein).
• Accordingly, to undertake such principles theAVDD12 can be established by some or all of the components shown inFIG. 1. For example, theAVDD12 can include one ormore displays14 that may be implemented by a high definition or ultra-high definition “4K” flat screen and that may be touch-enabled for receiving user input signals via touches on the display. TheAVDD12 may include one ormore speakers16 for outputting audio in accordance with present principles, and at least oneadditional input device18 such as e.g. an audio receiver/microphone for e.g. entering audible commands to theAVDD12 to control theAVDD12. Theexample AVDD12 may also include one or more network interfaces20 for communication over at least onenetwork22 such as the Internet, an WAN, an LAN, etc. under control of one ormore processors24. Thus, theinterface20 may be, without limitation, a Wi-Fi transceiver, which is an example of a wireless computer network interface. It is to be understood that theprocessor24 controls theAVDD12 to undertake present principles, including the other elements of theAVDD12 described herein such as e.g. controlling thedisplay14 to present images thereon and receiving input therefrom. Furthermore, note thenetwork interface20 may be, e.g., a wired or wireless modem or router, or other appropriate interface such as, e.g., a wireless telephony transceiver, or Wi-Fi transceiver as mentioned above, etc.
• In addition to the foregoing, theAVDD12 may also include one ormore input ports26 such as, e.g., a USB port to physically connect (e.g. using a wired connection) to another CE device and/or a headphone port to connect headphones to the AVDD12 for presentation of audio from the AVDD12 to a user through the headphones. TheAVDD12 may further include one or more tangible computerreadable storage medium28 such as disk-based or solid state storage. Also in some embodiments, theAVDD12 can include a position or location receiver such as but not limited to a cellphone receiver, GPS receiver and/oraltimeter30 that is configured to e.g. receive geographic position information from at least one satellite or cellphone tower and provide the information to theprocessor24 and/or determine an altitude at which theAVDD12 is disposed in conjunction with theprocessor24. However, it is to be understood that that another suitable position receiver other than a cellphone receiver, GPS receiver and/or altimeter may be used in accordance with present principles to e.g. determine the location of theAVDD12 in e.g. all three dimensions.
• Continuing the description of theAVDD12, in some embodiments theAVDD12 may include one ormore cameras32 that may be, e.g., a thermal imaging camera, a digital camera such as a webcam, and/or a camera integrated into theAVDD12 and controllable by theprocessor24 to gather pictures/images and/or video in accordance with present principles. Also included on theAVDD12 may be aBluetooth transceiver34 and other Near Field Communication (NFC)element36 for communication with other devices using Bluetooth and/or NFC technology, respectively. An example NFC element can be a radio frequency identification (RFID) element.
• Further still, theAVDD12 may include one or more auxiliary sensors37 (e.g., a motion sensor such as an accelerometer, gyroscope, cyclometer, or a magnetic sensor, an infrared (IR) sensor, an optical sensor, a speed and/or cadence sensor, a gesture sensor (e.g. for sensing gesture command), etc.) providing input to theprocessor24. TheAVDD12 may include still other sensors such as e.g. one or more climate sensors38 (e.g. barometers, humidity sensors, wind sensors, light sensors, temperature sensors, etc.) and/or one or morebiometric sensors40 providing input to theprocessor24. In addition to the foregoing, it is noted that theAVDD12 may also include an infrared (IR) transmitter and/or IR receiver and/orIR transceiver42 such as an IR data association (IRDA) device. A battery (not shown) may be provided for powering theAVDD12.
• Still referring toFIG. 1, in addition to theAVDD12, the system10 may include one or more other CE device types. In one example, afirst CE device44 may be used to control the display via commands sent through the below-described server while asecond CE device46 may include similar components as thefirst CE device44 and hence will not be discussed in detail. In the example shown, only twoCE devices44,46 are shown, it being understood that fewer or greater devices may be used.
• In the example shown, to illustrate present principles all threedevices12,44,46 are assumed to be members of an entertainment network in, e.g., a luxury suite of the stadium, or in a home, or at least to be present in proximity to each other in a location such as a house. However, for illustrating present principles thefirst CE device44 is assumed to be in the same room as theAVDD12, bounded by walls illustrated by dashedlines48.
• The example non-limitingfirst CE device44 may be established by any one of the above-mentioned devices, for example, a portable wireless laptop computer or notebook computer, and accordingly may have one or more of the components described below. Thesecond CE device46 without limitation may be established by a wireless telephone.
• Thefirst CE device44 may include one ormore displays50 that may be touch-enabled for receiving user input signals via touches on the display. Thefirst CE device44 may include one ormore speakers52 for outputting audio in accordance with present principles, and at least oneadditional input device54 such as e.g. an audio receiver/microphone for e.g. entering audible commands to thefirst CE device44 to control thedevice44. The examplefirst CE device44 may also include one or more network interfaces56 for communication over thenetwork22 under control of one or moreCE device processors58. Thus, theinterface56 may be, without limitation, a Wi-Fi transceiver, which is an example of a wireless computer network interface. It is to be understood that theprocessor58 controls thefirst CE device44 to undertake present principles, including the other elements of thefirst CE device44 described herein such as e.g. controlling thedisplay50 to present images thereon and receiving input therefrom. Furthermore, note thenetwork interface56 may be, e.g., a wired or wireless modem or router, or other appropriate interface such as, e.g., a wireless telephony transceiver, or Wi-Fi transceiver as mentioned above, etc.
• In addition to the foregoing, thefirst CE device44 may also include one ormore input ports60 such as, e.g., a USB port to physically connect (e.g. using a wired connection) to another CE device and/or a headphone port to connect headphones to thefirst CE device44 for presentation of audio from thefirst CE device44 to a user through the headphones. Thefirst CE device44 may further include one or more tangible computerreadable storage medium62 such as disk-based or solid state storage. Also in some embodiments, thefirst CE device44 can include a position or location receiver such as but not limited to a cellphone and/or GPS receiver and/oraltimeter64 that is configured to e.g. receive geographic position information from at least one satellite and/or cell tower, using triangulation, and provide the information to theCE device processor58 and/or determine an altitude at which thefirst CE device44 is disposed in conjunction with theCE device processor58. However, it is to be understood that that another suitable position receiver other than a cellphone and/or GPS receiver and/or altimeter may be used in accordance with present principles to e.g. determine the location of thefirst CE device44 in e.g. all three dimensions.
• Continuing the description of thefirst CE device44, in some embodiments thefirst CE device44 may include one ormore cameras66 that may be, e.g., a thermal imaging camera, a digital camera such as a webcam, and/or a camera integrated into thefirst CE device44 and controllable by theCE device processor58 to gather pictures/images and/or video in accordance with present principles. Also included on thefirst CE device44 may be aBluetooth transceiver68 and other Near Field Communication (NFC)element70 for communication with other devices using Bluetooth and/or NFC technology, respectively. An example NFC element can be a radio frequency identification (RFID) element.
• Further still, thefirst CE device44 may include one or more auxiliary sensors72 (e.g., a motion sensor such as an accelerometer, gyroscope, cyclometer, or a magnetic sensor, an infrared (IR) sensor, an optical sensor, a speed and/or cadence sensor, a gesture sensor (e.g. for sensing gesture command), etc.) providing input to theCE device processor58. Thefirst CE device44 may include still other sensors such as e.g. one or more climate sensors74 (e.g. barometers, humidity sensors, wind sensors, light sensors, temperature sensors, etc.) and/or one or morebiometric sensors76 providing input to theCE device processor58. In addition to the foregoing, it is noted that in some embodiments thefirst CE device44 may also include an infrared (IR) transmitter and/or IR receiver and/orIR transceiver78 such as an IR data association (IRDA) device. A battery (not shown) may be provided for powering thefirst CE device44.
• Thesecond CE device46 may include some or all of the components shown for theCE device44.
• Now in reference to the afore-mentioned at least oneserver80, it includes at least oneserver processor82, at least one tangible computerreadable storage medium84 such as disk-based or solid state storage, and at least onenetwork interface86 that, under control of theserver processor82, allows for communication with the other devices ofFIG. 1 over thenetwork22, and indeed may facilitate communication between servers and client devices in accordance with present principles. Note that thenetwork interface86 may be, e.g., a wired or wireless modem or router, Wi-Fi transceiver, or other appropriate interface such as, e.g., a wireless telephony transceiver.
• Accordingly, in some embodiments theserver80 may be an Internet server, and may include and perform “cloud” functions such that the devices of the system10 may access a “cloud” environment via theserver80 in example embodiments. Or, theserver80 may be implemented by a game console or other computer in the same room as the other devices shown inFIG. 1 or nearby.
• FIG. 2 shows anexample system100 in which first and second ultra high definition (UHD) displays102,104 are mounted on a wall, e.g., a wall of a home or a luxury stadium box. The UHD displays102,104 may be 4K displays. One or more control devices control presentation of the displays by sending commands wirelessly and/or over wired paths to one or more controllers. In the non-limiting example shown, acontroller106 controls thedisplays102,104, it being understood that a separate controller may be provided for each display. In the non-limiting example shown, content control on thefirst display102 is established by afirst control device108 while content control on thesecond display104 is established by asecond control device110, it being understood that a single control device may be used to establish control on both displays.
• Thecontrol devices108,110 may be, without limitation, portable computers such as tablet computers or laptop computers (also including notebook computers) or other devices with one or more of theCE device44 components shown inFIG. 1. Thedisplays102,104 may be monitors only and/or may include one or more of theprimary display14 components shown inFIG. 1. Thecontroller106 may be a personal computer (PC) or game console or server that contains one or more of the components variously shown inFIG. 1. In the non-limiting example shown, thecontrol devices108,110 communicate directly with thecontroller106 using, e.g., WiFi or Bluetooth; thecontrol devices108,110 do not communicate directly with thedisplays102,104. Instead, thecontroller106 communicates with thedisplays102,104 to establish presentation thereon in accordance with commands received from the control devices. It is to be understood that while thecontroller106 is shown physically separate from the displays inFIG. 2, it may be incorporated within the chassis of a display. As also shown, the displays may present plural contents inrespective content windows112.
• Thecontroller106 may receive video fromplural video cameras114. In the stadium context, afirst camera114 may image a first half of a sports field, racetrack, or other action venue whereas assecond camera114 may image the other half of the action venue, with the feeds from the two cameras being combined before being sent to thecontroller106 or combined by thecontroller106 and “stitched” to present a single video view of both halves of the action venue on one or both of thedisplays102,104. That is, the combined feed from both cameras may be presented on a single display in an 8K mode, or the combined feed may be spread across the juxtaposed displays such that one display shows one half of the action venue and the other display shows the other half. It will be appreciated that the feeds sent to the controller preferably are HD or more preferably UHD.
• As well, the cameras114 (through appropriate image processing down-resolution components) can present the same video feeds albeit at a lower resolution to thecontrol devices108,110. The UHD feeds may be sent to thecontroller106 over a network from a network address while the lower resolution feeds of the same content may be simultaneously sent to thecontrol devices108,110 over the network from the same or a different network address, such that the video content on the control devices is the same as the video content presentable on the displays, albeit typically of a lower resolution.
• Note that a dedicated local server or PS4 may not be required in some embodiments to manage the 4K and thumbnail feeds as well as analyze the commands coming from the tablet. Instead, this can happen in the cloud with the 4K TV and tablet having their own MAC address and the cloud server acting as though it were local to permit control of 4K monitors in remote locations as well.
• A location sensing system such as any of those described above may be used to determine where the control device is relative multiple 4K display locations to allow the user to roam and have the 4K content follow him. This provides for multiple 4K clusters in a stadium suite, each showing the same or different content. In this case what is showing on a particular 4K TV cluster can drive the UI on the tablet, or the other way around.
• InFIG. 3, a control device such as thecontrol device108 shown inFIG. 2 presents on its display a user interface (UI)116 of a user interface (UI) presenting a video image of content and aborder118 superimposed on aportion120 of the video image which is smaller than the video image as shown. For illustration, assume the video image inFIG. 3 is of a football game, with “Os” representing offensive players and “Xs” representing defensive players.
• When a user has instantiated the border118 (by, e.g., selecting a “pan and zoom” selector122), the control device in response sends a command to the controller to cause a large display such thedisplay102 inFIG. 2 to present on thedisplay102 only the (higher definition) portion of the content enclosed in theborder118 on thecontrol device108. In the illustration shown, the user has positioned theborder118 on thecontrol device108 over two offensive players and two defensive players with subscripts “1” to distinguish them from the other player symbols in the figure. In response, the control device, as indicated by thearrow124, has commanded the controller to present on thedisplay device102 only the content enclosed by the border on the control device, in the example shown, to present only the two offensive players and two defensive players with subscripts “1”. It will readily be appreciated that the controller further has zoomed the video presentation on the demanded portion to substantially fill the entire screen of thedisplay device102.
• In one example, the screen of thecontrol device108 is a touch screen display, and a user may touch theborder118 and/or portion enclosed thereby and drag (as indicated by the arrow126) the border to a new portion of the video as indicated by the dottedline box128, releasing the user touch once the border has been dragged to the desired part of the video shown on the control device. In thenew portion128, two defensive players “X” are shown, denoted by subscripts “2” to distinguish them. As indicated by thearrow130, this drag and drop causes the controller to pan the zoomed video from the first portion to the second portion in the direction of the drag until the second portion of the (higher definition) video substantially fills the screen of thedisplay device102 as shown at132 in the figure.
• Thus, responsive to the drag and drop of theborder118 on thecontrol device108, the content related to the video image on thedisplay device102 is entirely established, in temporal sequence, by a zoomed presentation of the first portion, then a moving pan across at least part of the video image on the display device in concert with the user input to move the border to the second portion of the video image on the control device, to end at a zoomed presentation on the display device of the second portion. During the drag and drop process, the control device presents both the entire video image of the content and theborder118 superimposed on theportion120 of the video image as the user input causes the border to move across the video image of the content, whereas thedisplay device102 is caused to present only content from the video image corresponding to content120 within theborder118 on the control device.
• HTML5 may be used along with JavaScript (including some JavaScript libraries), and CSS in one implementation. Video files may be stored locally on the control device and played in the browser using the video tag of HTML5. Live streaming files from a local streaming server, streaming files from internet and live tuner signal can also be used as the source. To select a different file, a user drags and drops a tile; based on the id of the tile, the path of the video in the quad portion (on which the tile id dropped) of the display presenting video in four quadrants selected is changed to the correct video and this new video is played. A full screen API may not be used since it requires user interaction to allow full screen on the control device. Accordingly, as a workaround for full screen, all videos can be paused, then the video selected can be scaled by the browser to 4K resolution. If a 4K file is present, the 4K file is used, then no browser scaling is needed. Websocket may be used to communicate through IP from the control device to the controller to control the display device. Messages may be broadcast to all the display devices, then each display device browser can use the message it needs. Drag and drop can be done using the jQuery UI library, and scrolling can be done using CSS position updating. The stitch image zoom effect can be done by drawing video on the HTML5 canvas, sending coordinates from control device to the controller so the controller knows which portion of the video to zoom on in the display device.
• A phone application may also be implemented in HTML5, allowing audio files from the server to be played on a speaker, e.g., of the display device or other device, through IP. The phone application audio matches the audio for the four videos played in the quad view, and each audio file can be selected for playback. When selecting an external device connected to a different HDMI input of the display device (such as video disk player, a satellite feed, etc.), when a user drags the appropriate tile for the external device, the control device may send IP commands to the display device (via the controller) to change input. If a tile corresponding to a video is drag and dropped, another IP command can be sent to the display device (via the controller) to change input back to PC and/or controller and the video file selected is played from the PC and/or controller.
• FIGS. 4-7 illustrate additional principles that may be used according to present principles. On the left side ofFIG. 4, one of thedisplay devices102 fromFIG. 2 and one of thecontrol devices108 are shown schematically, and on the right side ofFIG. 4 the same devices are shown schematically after an operation. As shown on the left side ofFIG. 4, a multi-view channel has been launched on thedisplay device102, resulting in four video windows400 (“quad view”) being presented on thedisplay device102. Eachwindow400 presents a video stream from a respective audio video (AV) program P1-P4 from an Internet or broadcast source (cable, satellite, etc.) The audio from only one of the AV programs is played by the display device. Thewindows400 may be substantially identical in size if desired, as shown.
• AUI402 is presented on the display of thecontrol device108. As shown, theUI402 includes pluralmain selectors404 arranged in a layout, preferably the same layout as thewindows400 on thedisplay device102 as shown. Eachmain selector404 is established by a respective video feed, in the example shown, the same content albeit perhaps in lower resolution as the four videos in the quad view of thedisplay device102, as duly indicated by use of the same video program designators P1-P4.
• TheUI402 may further include arow406 ofadditional content selectors408 apart from the programs P1-P4 shown in themain selectors404, although in the embodiment shown, for ease of disclosure, the same four programs P1-P4 establish the first fourcontent selectors408 in therow406, while the last two content selectors indicate they may be selected to present content from two additional programs P5 and P6. In some embodiments, unlike themain selectors404, which recall are established by moving video, thecontent selectors408 in therow406 may be established by still image thumbnails.
• Furthermore, acolumn410 ofaudio selectors412 may be presented on theUI402. Eachvolume selector412 in thecolumn410 may correspond to a respective content in thecontent selectors408 in therow406. Eachaudio selector412 may include a respective audio on/offsymbol414, with all of thesymbols414 except one having a line through them indicating that the audio represented by those selectors is not being played on thedisplay device102. In contrast, in the example shown thesymbol414 of thetop audio selector412 does not have a line through it, indicating that the audio from the program associated with the top selector, in the example shown, program P1, is being played on thedisplay device102. Touching anaudio selector412 on thecontrol device108 causes the control device to command the controller to switch audio play on thedisplay device102 to the audio represented by the touched audio selector on thecontrol device108. This also causes the line through therespective symbol414 of the touched selector to be removed, and a line placed onto thesymbol414 of theselector412 representing the replaced audio.
• The right side ofFIG. 4 illustrates the dragging and dropping one of thecontent selectors408 onto amain selector404 changes the video in thatmain selector404 to the video represented by the dragged and droppedcontent selector408. Not only does theUI402 on thecontrol device108 thus change, but also, as indicated in the top right portion ofFIG. 4, the video presented in the window400 (in this, the top left window) of thedisplay device102 is also caused to change to the video represented by the dragged and droppedcontent selector408. This may be done by thecontrol device108, responsive to the drag and drop, obtaining the network address or channel number of the video represented by the dragged and droppedcontent selector408 and commanding the controller to present video from that network address in thewindow400 corresponding to themain selector404 onto which thecontent selector408 was dragged and dropped. In any case, it will readily be appreciated that themain selectors404 on thecontrol device108 mirror thewindows400 on thedisplay device102. A user may move his hand left or right on therow406 of content selectors to cause selectors for additional content to scroll onto the display of the control device. The new content typically includes additional program channels or Internet content related to the theme of the programming presented on thedisplay device102, including, for example, sports statistics related to a sporting event in one of thewindows400.
• The left side ofFIG. 5 illustrates that “throwing” one of themain selectors404 on thecontrol device108 “to” thedisplay device102 causes the display device to switch to a full screen presentation500 (shown on the right ofFIG. 5) of the content represented by the “thrown”main selector404. The processor of thecontrol device108 may infer that amain selector404 has been “thrown” by a user dragging the main selector upwards toward the top of the control device, responsive to which thecontrol device108 sends a command to the controller to present the associated content full screen on thedisplay device102.
• Although not shown inFIG. 5, commanding thedisplay device102 into the full screen mode as described above may result in themain selectors404 on thecontrol device108 merging into a single large selector with the same content shown full screen on the display device. However, therow406 ofcontent selectors406 andcolumn410 ofaudio selectors412 can remain unchanged. This single large main selector in the area formerly occupied by the four main selectors may be touched to cause the display device (pursuant to a command from the control device) to revert to the quad view shown on the left side ofFIG. 5 and to also cause themain selectors404 on thecontrol device108 to mirror the display device views, in this case, to resume the four main selector quad view shown on the bottom left ofFIG. 4.
• Alternatively, throwing amain selector404 to cause the display device to enter full screen mode as described may not alter the appearance of themain selectors404, which can remain in the quad view shown on the control device. Subsequently touching any one of themain selectors404 on the control device may result in the control device commanding the controller to resume the quad view presentation on the display device. Or, if desired, as shown on the bottom right ofFIG. 5, when the control device has been configured to command the display device to enter full screen mode, with the fourmain selectors404 remaining on the control device UI, touching one of the main selectors may cause the corresponding content to be presented full screen on the control device. The content presented full screen on the display device may be the same or different than the content presented full screen on the control device, depending on whatmain selector404 was thrown to the display device and whatmain selector404 subsequently was touched by a user. A subsequent touch anywhere on the control device screen may cause the control device and display device to resume the layouts shown on the left side ofFIG. 5.
• To view the details of any content represented by acontent selector408, as shown in the left side ofFIG. 6 a user may simply touch thecontent selector408. This may cause adetail screen600 to appear on the control device108 (but not on the display device102), so that a person controlling presentation on the display device by means of thecontrol device108 can observe the detailed information about content prior to presenting the content (or information about the content) on thedisplay device102.
• FIG. 7 is a series of screen shots on thecontrol device108 to illustrate further optional details. Aremote control selector702 may be presented which if touched as indicated at704 can cause a remote-control likepresentation706 to be presented on thecontrol device108. Thepresentation706 emulates a standard remote control, with alpha-numeric touch-enabled input selectors, channel and volume up/down selectors, and cursor arrow selectors as shown. Touching theselector702 again causes the initial presentation (upper right panel inFIG. 7) to resume on the control device.
• When asingle control device108 is used to control bothdisplay device102 shown inFIG. 2, as shown at708 inFIG. 7main selectors710 corresponding to the quad view of a first one of thedisplay devices102 may be presented and used to control that display device according to principles above. Also, a smaller quad view of alternatemain selectors712 may be presented, representing content being presented on thesecond display device102. To enable control of thesecond display device102 using thecontrol device108, a user need only touch the smaller quad view of alternatemain selectors712 as shown at714, causing the smaller quad view of alternatemain selectors712 to animate to anenlarged configuration716 and the quad view of main selectors corresponding to the first one of thedisplay devices102 to animate to become smaller in size as shown at718. The enlarged configuration of main selectors on thecontrol device108 appertaining to thesecond display device102 may then be used to control the presentation on the second display device according to principles above.
• The bottom two screen shots inFIG. 7 show that tapping720 on a content selector can cause adetail presentation722 to be shown on the control device, showing the details of the content represented by the tapped content selector. Also, as shown at724, instead of a column of audio selectors as described previously, respective audio on/off symbols analogous to thesymbols414 inFIG. 4 may be presented on eachmain selector726 and if touched cause the control device to command the controller to replace the associated audio with the touched symbol to replace the audio currently being played on the display device being controlled.
• While a four screen quad view is discussed and shown, any number of windows in a multi-window arrangements may be used.
• While the particular CONTROL OF LARGE SCREEN DISPLAY USING WIRELESS PORTABLE COMPUTER INTERFACING WITH DISPLAY CONTROLLER is herein shown and described in detail, it is to be understood that the subject matter which is encompassed by the present invention is limited only by the claims.

Claims (20)

What is claimed is:

1. A control device comprising:

at least one computer readable storage medium bearing instructions executable by a processor;

at least one processor configured for accessing the computer readable storage medium to execute the instructions to configure the processor for:

presenting on a display of the control device a user interface (UI) having plural selectors arranged in a layout, each selector being established by a respective video feed;

receiving a first user input corresponding to a first command;

wirelessly transmitting the first command to a controller of the display device to command the controller to present plural video feeds on the display device, the video feeds on the control device being of the same content as respective video feeds on the display device;

receiving a second user input corresponding to a second command; and

wirelessly transmitting the second command to the controller to command the controller to present a single video feed full screen on the display device with the layout of the selectors on the control device remaining unchanged.

2. The control device ofclaim 1, wherein the first user input is established by a user touch on any portion of a display of the control device.

3. The control device ofclaim 1, wherein the second user input is established only by a user touch on a selector corresponding to the single video feed.

4. The control device ofclaim 1, wherein the processor when executing the instructions is further configured for:

transmitting the first command to the controller of the display device to command the controller to present the plural video feeds on the display device in a same layout as the video feeds establishing the selectors are arranged on the display of the monitoring device.

5. The control device ofclaim 1, wherein the processor when executing the instructions is further configured for:

receiving a third user input corresponding to a third command, the third user input being a drag of an image associated with a content selector and a drop of the image onto a portion of the control device, the third command commanding the controller to cause the display device to present a video corresponding to the content selector in a portion of the display device that corresponds to the portion of the control device onto which the image was dropped.

6. The control device ofclaim 1, wherein the control device is established by a portable computer.

7. The control device ofclaim 1, wherein the processor when executing the instructions is further configured for:

providing a visible indicator on the control device that the display device has been commanded to present the single video feed full screen on the display device.

8. System comprising:

display device configured for presenting plural video contents in respective windows of the display device;

controller configured for controlling the display device; and

control device configured for communicating commands to the controller to control presentation on the display device, the control device being configured to present a user interface (UI) having plural selectors arranged in a layout on a screen of the control device, each selector being established by a respective video feed having a first resolution, the control device being configured for causing the display device to present plural video feeds on the display device, the video feeds on the control device being of the same content as respective video feeds on the display device, the respective video feeds on the display device having second resolution higher than the first resolution, the control device being configured for causing a single video feed full screen to be presented on the display device with the layout of the selectors on the control device remaining unchanged.

9. The system ofclaim 8, wherein the video feeds on the display device are received from a source and the video feeds on the control device are also received from the source.

10. The system ofclaim 8, wherein the control device is configured for causing the single video feed full screen to be presented on the display device responsive to a user touch on any portion of a display of the control device.

11. The system ofclaim 8, wherein the control device is configured for causing the single video feed full screen to be presented on the display device responsive to only a user touch on a selector corresponding to the single video feed.

12. The system ofclaim 8, wherein the control device is further configured for:

transmitting a command to the controller of the display device to command the controller to present the plural video feeds on the display device in a same layout as the video feeds establishing the selectors are arranged on the display of the monitoring device.

13. The system ofclaim 8, wherein the control device is further configured for:

receiving a third user input corresponding to a third command, the third user input being a drag of an image associated with a content selector and a drop of the image onto a portion of the control device, the third command commanding the controller to cause the display device to present a video corresponding to the content selector in a portion of the display device that corresponds to the portion of the control device onto which the image was dropped.

14. The system ofclaim 8, wherein the control device is established by a portable computer.

15. The system ofclaim 8, wherein the control device is further configured for:

providing a visible indicator on the control device that the display device has been commanded to present the single video feed full screen on the display device.

16. The system ofclaim 8, wherein the display device is established by an ultra high definition (UHD) display.

17. The system ofclaim 8, wherein the controller is established by a game console.

18. The system ofclaim 8, wherein the controller is established by a personal computer (PC).

19. The system ofclaim 8, wherein the control device is established by a laptop computer or tablet computer.

20. Method comprising:

sending plural video feeds having a first resolution to a control device over a computer network;

sending plural video feeds having a second resolution to a display device over the computer network, the second resolution being higher than the first, the video feeds of the first resolution being the same content as the video feeds of the second content, wherein video presentation on the display device is controlled at least in part based on user touches of the video feeds presented on the control device.

Images