US20160133052A1 - Virtual environment for sharing information - Google Patents

Abstract

An electronic device providing information through a virtual environment is disclosed. The device includes: a display; and an information providing module functionally connected with the display, wherein the information providing module displays an object corresponding to an external electronic device for the electronic device through the display, obtains information to be output through the external electronic device, and provides contents corresponding to the information in relation to a region, on which the object is displayed.

Description

CROSS-REFERENCE TO RELATED APPLICATION
• This application is based on and claims priority under 35 U.S.C. §119 to Korean Patent Application No. 10-2014-0154367, filed Nov. 7, 2014, which is incorporated by reference herein in its entirety.
BACKGROUND
• 1. Field
• The disclosure relates to an electronic device, and for example, to an apparatus and a method for sharing information through a virtual environment.
• 2. Description of Prior Art
• Recently, electronic devices have been developed into various forms, such as a portable device (for example, a smart phone and a tablet PC) or a wearable device (for example, a smart watch and a head-mounted device)), as well as a smart home appliance (for example, a smart television and a game console). The electronic devices having various forms may exchange information. For example, a smart television, a smart phone, and a notebook computer may be connected with each other through a network. The smart phone may transmit a picture stored in a smart phone to a smart television. In this case, a user of the smart television may check the picture stored in the smart phone through the smart television. Further, a notebook computer may transmit a movie to the smart television. In this case, the user of the smart television may watch the movie, instead of the picture, through the smart television
• According to the related art, an electronic device may provide a user with contents obtained from an external electronic device. In this case, the electronic device may provide the contents without discriminating the contents from other previously obtained contents. Accordingly, it may be difficult for a user to discriminate the contents obtained from the external electronic device from other contents. For example, it is difficult for a user to recognize information about a device, from which the contents are obtained.
• When the electronic device obtains a plurality of contents from a plurality of external electronic devices, respectively, the electronic device may not simultaneously provide the user with the plurality of contents in the related art. Accordingly, the user is inconvenienced by needing to first check one of the plurality of contents, and then check other contents.
• The electronic device may provide a user with various virtual environments so that the user may view a 2D or 3D image. For example, the electronic device may provide a user with a new environment regardless of a space, in which a user is actually located. Accordingly, the user may more vividly view the 2D or 3D image based on a virtual environment.
• However, the electronic device in the related art cannot reflect a space, in which a user is actually located, to a virtual environment. For example, since the electronic device is capable of using only a set image, the electronic device cannot represent various electronic devices (for example, a television, a refrigerator, or a computer) located in an actual space on a virtual environment. Accordingly, when a user uses a virtual environment, the user may not directly control an electronic device located in an actual space.
SUMMARY
• An example of the disclosure provides an electronic device and a method capable of relating contents obtained from an external electronic device to an object corresponding to the external electronic device through a virtual environment and providing the obtained contents. Accordingly, the electronic device may discriminate the contents obtained from the external electronic device from other contents and provide the obtained contents through the virtual environment.
• Another example of the disclosure provides an electronic device and a method capable of relating contents obtained from a plurality of external electronic devices to a plurality of objects corresponding to the plurality of external electronic devices, respectively, and simultaneously providing the obtained contents.
• Another example of the disclosure provides an electronic device and a method capable of enabling a user to directly control electronic devices disposed in an actual space through a virtual environment by reflecting a space, in which the user is actually located, to the virtual environment.
• In accordance with an example of the disclosure, an electronic device is provided. The electronic device includes: a display; and an information providing module in the form of processing circuitry functionally connected with the display, wherein the information providing module is configured to display an object corresponding to an external electronic device for the electronic device through the display, to obtain information to be output through the external electronic device, and to provide contents corresponding to the information in relation to a region, on which the object is displayed.
• The electronic device and the method according to various examples may provide, for example, contents obtained from an external electronic device to a user in relation to an object corresponding to the external electronic device, thereby enabling the user to discriminate the contents obtained from the external electronic device from other contents based on the object.
• The electronic device and the method according to various examples may simultaneously provide, for example, a plurality of contents obtained from a plurality of external electronic devices through a plurality of objects corresponding to the plurality of external electronic devices, respectively, thereby reducing or overcoming inconvenience caused when a user needs to check each of the plurality of contents.
• The electronic device and the method according to various examples may provide, for example, contents obtained through an external electronic device through a virtual environment, thereby providing a service linked with the external electronic device through the virtual environment.
BRIEF DESCRIPTION OF THE DRAWINGS
• The above and other objects, features and advantages of the disclosure will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which like reference numerals refer to like elements, and wherein:
• FIG. 1 illustrates an example use environment of a plurality of electronic devices;
• FIG. 2 is a block diagram illustrating an example electronic device;
• FIG. 3 is a block diagram illustrating an example electronic device;
• FIG. 4 illustrates an example of an actual use environment, in which an electronic device is linked with an external electronic device;
• FIG. 5 illustrates an example of a virtual environment provided by an electronic device;
• FIG. 6 is a block diagram illustrating an example information providing module;
• FIG. 7 illustrates an example of a user interface provided through an electronic device;
• FIG. 8 illustrates an example of a user interface provided through the electronic device;
• FIG. 9 illustrates an example of a user interface provided through the electronic device;
• FIG. 10 illustrates an example of a user interface provided through the electronic device;
• FIG. 11 illustrates an example of a user interface provided through the electronic device; and
• FIG. 12 is a flowchart illustrating an example method for providing information by an electronic device.
DETAILED DESCRIPTION
• Hereinafter, the disclosure will be described with reference to the accompanying drawings. Although example embodiments are illustrated in the drawings and related detailed descriptions are discussed in the disclosure, the disclosure may have various modifications and several examples. However, various examples of the disclosure are not limited to an example implementation form and it should be understood that the disclosure includes all changes and/or equivalents and substitutes included in the spirit and scope of various examples of the disclosure. In connection with descriptions of the drawings, similar components are designated by the same reference numeral.
• The term “include” or “may include” which may be used in describing various examples of the disclosure refers to the existence of a corresponding disclosed function, operation or component which can be used in various examples of the disclosure and does not limit one or more additional functions, operations, or components. In various examples of the disclosure, the terms such as “include” or “have” may be construed to denote a certain characteristic, number, step, operation, constituent element, component or a combination thereof, but may not be construed to exclude the existence of or a possibility of addition of one or more other characteristics, numbers, steps, operations, constituent elements, components or combinations thereof.
• In various examples of the disclosure, the expression “or” or “at least one of A or/and B” includes any or all of combinations of words listed together. For example, the expression “A or B” or “at least A or/and B” may include A, may include B, or may include both A and B.
• The expression “1”, “2”, “first”, or “second” used in various examples of the disclosure may modify various components of the various examples but does not limit the corresponding components. For example, the above expressions do not limit the sequence and/or importance of the components. The expressions may be used for distinguishing one component from other components. For example, a first user device and a second user device indicate different user devices although both of them are user devices. For example, without departing from the scope of the disclosure, a first structural element may be referred to as a second structural element. Similarly, the second structural element also may be referred to as the first structural element.
• When it is stated that a component is “coupled to” or “connected to” another component, the component may be directly coupled or connected to another component or another component may exist between the component and another component. In contrast, when it is stated that a component is “directly coupled to” or “directly connected to” another component, a component does not exist between the component and another component.
• The terms used in describing various examples of the disclosure are merely examples for describing a given embodiment but do not limit the various examples of the disclosure. Singular forms are intended to include plural forms unless the context clearly indicates otherwise.
• Unless defined differently, all terms used herein, which include technical terminologies or scientific terminologies, have the same meaning as that understood by a person skilled in the art to which the disclosure belongs. Such terms as those defined in a generally used dictionary are to be interpreted to have the meanings equal to the contextual meanings in the relevant field of art, and are not to be interpreted to have ideal or excessively formal meanings unless clearly defined in the description.
• An electronic device according to examples of the disclosure may be a device including a communication function. For example, the electronic device may be one or a combination of a smart phone, a tablet Personal Computer (PC), a mobile phone, a video phone, an e-book reader, a desktop PC, a laptop PC, a netbook computer, a Personal Digital Assistant (PDA), a camera, a wearable device (for example, a Head-Mounted-Device (HMD) such as electronic glasses, electronic clothes, and electronic bracelet, an electronic necklace, an electronic appcessary, an electronic tattoo, and a smart watch.
• According to some examples, the electronic device may be a smart home appliance having a communication function. The smart home appliance may include at least one of a TeleVision (TV), a Digital Video Disk (DVD) player, an audio player, an air conditioner, a cleaner, an oven, a microwave oven, a washing machine, an air cleaner, a set-top box, a TV box (for example, Samsung HomeSync™, Apple TV™, or Google TV™), game consoles, an electronic dictionary, an electronic key, a camcorder, and an electronic frame.
• According to some examples, the electronic device may include at least one of various types of medical devices (for example, Magnetic Resonance Angiography (MRA), Magnetic Resonance Imaging (MRI), Computed Tomography (CT), a scanner, an ultrasonic device and the like), a navigation device, a Global Positioning System (GPS) receiver, an Event Data Recorder (EDR), a Flight Data Recorder (FDR), a vehicle infotainment device, electronic equipment for a ship (for example, a navigation device for ship, a gyro compass and the like), avionics, a security device, a head unit for a vehicle, an industrial or home robot, an Automatic Teller Machine (ATM) of financial institutions, and a Point Of Sale (POS) device of shops.
• According to some examples, the electronic device may include at least one of furniture or a part of a building/structure, an electronic board, an electronic signature receiving device, a projector, and various types of measuring devices (for example, a water meter, an electricity meter, a gas meter, a radio wave meter and the like) including a camera function. The electronic device according to various examples of the disclosure may be one or a combination of the above described various devices. Further, the electronic device according to various examples of the disclosure may be a flexible device. It will be apparent to those skilled in the art that the electronic device according to various examples of the disclosure is not limited to the above described devices.
• Hereinafter, an electronic device according to various examples of the disclosure will be described with reference to the accompanying drawings. The term “user” used in various examples may refer to a person who uses an electronic device or a device (for example, an artificial intelligence electronic device) which uses an electronic device.
• According to an example of the disclosure, a screen of an electronic device may be split into at least two windows according to a predefined split manner and displayed through a display of an electronic device. The windows may be referred to as split windows. According to an example, the split windows refer to windows displayed on a display of an electronic display not to be superposed one on another.
• According to an example, a popup window may refer to a window displayed on a display of an electronic device to hide or to be superposed on a portion of a screen under execution.
• According to an example of the disclosure, an electronic device using a split window and a popup window is capable of displaying two or more application execution screens or function execution screens. Thus, the split windows and the popup window may be referred to as a multi-window.
• An electronic device according to various examples will be described with reference to the accompanying drawings. As used herein, the term “user” may indicate a person who uses an electronic device or a device (e.g., an artificial intelligence electronic device) that uses an electronic device. Anelectronic device101 within anetwork environment100 according to various examples will be described with reference toFIG. 1. Theelectronic device101 may include aninformation providing module110 in the form of processing circuitry, abus120, aprocessor130, amemory140, an input/output interface160, adisplay170, and acommunication interface180. In some examples, theelectronic device101 may omit at least one of the above elements or may further include other elements.
• Theinformation providing module110 may be configured to process at least some of the information obtained from other elements (for example, theprocessor130, thememory140, the input/output interface160, and the communication interface180), and provide the user with the obtained information by various methods. For example, the information providing module orprocessing circuitry110 may be configured to display an object corresponding to an external electronic device (for example, theelectronic devices102 and104 or the server106) through thedisplay170 functionally connected with theelectronic device101 using theprocessor130 or independently from theprocessor130. According to an example, the object may have an image corresponding to an appearance of an external electronic device.
• According to an example, theinformation providing module110 may provide information outputtable by an external electronic device (for example, the electronic device102) through at least one of the object corresponding to the external electronic device displayed on thedisplay170 or the external electronic device depending on whether theelectronic device101 is detached from or attached to the user. For example, the external electronic device may be a television. Theinformation providing module110 may provide (for example, display) at least some of information (for example, an image, text, a sound, or notification information) outputtable through the television on a partial area of thedisplay170, on which the object having a television image corresponding to the television is displayed. Additional information associated with theinformation providing module110 will be described below in conjunction withFIGS. 4 to 12.
• Thebus120 may be a circuit connecting the above described components and transmitting communication (for example, a control message) between the above described components.
• Theprocessor130 may include one or more a CPU (central processing unit (CPU)), application processors (application processor (AP)), or communication processors (communication processor (CP)). TheProcessor130, for example, may be configured to execute the operation or data processing related to control and/or communication of at least one the other components of theelectronic device101.
• Thememory140 stores commands or data received from theprocessor130 or other components of theelectronic device101 or generated by theprocessor130 or other components of theelectronic device101. Thememory140 may includeprogramming modules150, for example, akernel151,middleware153, an Application Programming Interface (API)155, and anapplication157. Each of the aforementioned programming modules may be implemented by software, firmware, hardware, or a combination of two or more thereof. Thekernel151 controls or manages system resources (for example, thebus120, theprocessor130, or the memory140) used for executing an operation or function implemented by the remaining other programming modules, for example, themiddleware153, theAPI155, or theapplication157. Further, thekernel151 provides an interface for accessing individual components of theelectronic device101 from themiddleware153, theAPI155, or theapplication157 to control or manage the components. Themiddleware153 performs a relay function of allowing theAPI155 or theapplication157 to communicate with thekernel151 to exchange data. Further, in operation requests received from theapplication157, themiddleware153 performs a control for the operation requests (for example, scheduling or load balancing) by using a method of assigning a priority, by which system resources (for example, thebus120, theprocessor130, thememory140 and the like) of theelectronic device101 can be used, to theapplication157.
• TheAPI155 may, for example, be an interface by which theapplication157 can control a function provided by the kernel141 or themiddleware153 and includes, for example, at least one interface or function (for example, command) for a file control, a window control, image processing, or a character control
• The input/output interface160 can receive, for example, a command and/or data from a user, and transfer the received command and/or data to theprocessor130 and/or thememory140 through thebus120. Examples of thedisplay170 may include a Liquid Crystal Display (LCD), a Light-Emitting Diode (LED) display, an Organic Light-Emitting Diode (OLED) display, a MicroElectroMechanical Systems (MEMS) display, and an electronic paper display, etc. Thedisplay170, for example, may display various contents (e.g., text, images, videos, icons, or symbols) to the user. Thedisplay170 may include a touch screen and receive, for example, a touch, gesture, proximity, or hovering input using an electronic pen or the user's body part.
• Thecommunication interface180 may enable communication, for example, between theelectronic device101 and an external device (e.g., the first externalelectronic device102, the second externalelectronic device104, or the server106). For example, thecommunication interface180 may be connected to anetwork162 through wireless or wired communication to communicate with the external device (e.g., the second externalelectronic device104 or the server106).
• The wireless communication may use at least one of, for example, Long Term Evolution (LTE), LTE-Advance (LTE-A), Code Division Multiple Access (CDMA), Wideband CDMA (WCDMA), Universal Mobile Telecommunications System (UMTS), WiBro (Wireless Broadband), and Global System for Mobile Communications (GSM), as a cellular communication protocol. In addition, the wireless communication may include, for example,short range communication164. The short-range communication164 may include at least one of, for example, Wi-Fi, Bluetooth, Near Field Communication (NFC), infrared communication, and Global Positioning System (GPS). The wired communication may include at least one of, for example, a Universal Serial Bus (USB), a High Definition Multimedia Interface (HDMI), Recommended Standard 232 (RS-232), and a Plain Old Telephone Service (POTS). Thenetwork162 may, for example, include at least one of communication networks, such as a computer network (e.g., a LAN or a WAN), the Internet, and a telephone network.
• At least one of the first and second externalelectronic devices102 and104 may be the same or a different type of device from theelectronic device101. According to an example, theserver106 may include a group of one or more servers. According to various examples, all or some of the operations performed in theelectronic device101 may be performed in another electronic device or a plurality of electronic devices (e.g., theelectronic devices102 and104 or the server106). According to an example, when theelectronic device101 has to perform some functions or services automatically or in response to a request, theelectronic device101 may make a request for performing at least some functions relating thereto to another device (e.g., theelectronic device102 or104 or the server106) instead of performing the functions or services by itself.
• The other electronic device (e.g., theelectronic device102 or104 or the server106) may carry out the requested functions or the additional functions and transfer the result to theelectronic device101. Theelectronic device101 may process the received result as it is or additionally provide the requested functions or services. To this end, for example, cloud computing, distributed computing, or client-server computing technology may be used.
• FIG. 2 is a block diagram illustrating an exampleelectronic device201. Theelectronic device201 may, for example, include a whole or a part of theelectronic device101 illustrated inFIG. 1. Referring toFIG. 2, theelectronic device201 may, for example, include one or more Processors (for example, Application Processors (APs))210, acommunication module220, a Subscriber Identification Module (SIM)card224, amemory230, asensor module240 including various sensors, aninput device250, adisplay260, aninterface270, anaudio module280, acamera module291, apower managing module295, abattery296, an indicator297, and amotor298.
• Theprocessor210 operates an operating system (OS) or an application program so as to control a plurality of hardware or software component elements connected to theprocessor210 and execute various data processing and calculations including multimedia data. Theprocessor210 may be implemented by, for example, a System on Chip (SoC). According to an example, theprocessor210 may further include a Graphic Processing Unit (GPU). Thecommunication module220 may have a configuration the same or similar to that of thecommunication interface180 ofFIG. 1. Thecommunication module220 may include, for example, acellular module221, a Wi-Fi module223, aBluetooth module225, aGPS module227, anNFC module228, and a Radio Frequency (RF)module229.
• Thecellular module221 may provide a voice call, image call, a text message service, or an Internet service through, for example, a communication network. According to an example, thecellular module221 may distinguish and authenticate theelectronic device201 within a communication network by using a subscriber identification module224 (for example, the SIM card). According to an example of the disclosure, thecellular module221 may perform at least some of the functions that theprocessor210 may provide. According to an example, thecellular module221 may include a Communication Processor (CP).
• At least one of the Wi-Fi module223, theBluetooth module225, theGPS module227, and theNFC module228 may be, for example, a processor for processing data transceived through a corresponding module. According to an example, at least some (two or more) of thecellular module221, the Wi-Fi module223, theBluetooth module225, theGPS module227, and theNFC module228 may be included in one Integrated Chip (IC) or IC package.
• TheRF module229 may transmit/receive, for example, a communication signal (for example, an RF signal). TheRF module229 may include, for example, a transceiver, a Power Amplifier Module (PAM), a frequency filter, a Low Noise Amplifier (LNA), or an antenna. According to another example, at least one of thecellular module221, the Wi-Fi module223, theBT module225, theGPS module227, and theNFC module228 may transmit/receive an RF signal through a separate RF module.
• Thesubscriber identification module224 may include, for example, a card including a subscriber identification module and/or an embedded SIM, and may contain unique identification information (e.g., an Integrated Circuit Card Identifier (ICCID)) or subscriber information (e.g., an International Mobile Subscriber Identity (IMSI)).
• The memory230 (for example, the memory140) may include, for example, aninternal memory232 or anexternal memory234. Aninternal memory232 may include at least one of a volatile memory (for example, a Dynamic Random Access Memory (DRAM), a Static RAM (SRAM), a Synchronous Dynamic RAM (SDRAM), and the like, and a non-volatile memory (for example, a One Time Programmable Read Only Memory (OTPROM)), a Programmable ROM (PROM), an Erasable and Programmable ROM (EPROM), an Electrically Erasable and Programmable ROM (EEPROM), a mask ROM, a flash ROM, a flash memory (for example, a NAND flash memory or a NOR flash memory), a hard disc drive, a Solid State Drive (SSD), and the like.
• Anexternal memory234 may further include a flash drive, for example, a Compact Flash (CF), a Secure Digital (SD), a Micro Secure Digital (Micro-SD), a Mini Secure Digital (Mini-SD), an eXtreme Digital (xD), a Multi Media Card (MMC), a memory stick, or the like. Theexternal memory234 may be functionally and/or physically connected to theelectronic device201 through various interfaces.
• Thesensor module240 may, for example, include various sensors configured to measure a physical quantity or detect an operation state of theelectronic device201, and may convert the measured or detected information into an electrical signal. For example, thesensor module240 may include at least one of agesture sensor240A, agyro sensor240B, anatmospheric pressure sensor240C, a magnetic sensor240D, anacceleration sensor240E, agrip sensor240F, aproximity sensor240G, acolor sensor240H (for example, a Red/Green/Blue (RGB) sensor), a bio-sensor240I, a temperature/humidity sensor240J, anillumination sensor240K, and an Ultra Violet (UV)sensor240M. Additionally or alternatively, thesensor module240 may include, for example, an E-nose sensor, an electromyography (EMG) sensor, an electroencephalogram (EEG) sensor, an electrocardiogram (ECG) sensor, an Infrared (IR) sensor, an iris sensor, and/or a fingerprint sensor. Thesensor module240 may further include a control circuit for controlling one or more sensors included therein. In some examples of the disclosure, anelectronic device201 may further include a processor configured to control thesensor module240 as a part of or separately from theprocessor210, and may control thesensor module240 while theprocessor210 is in a sleep state.
• Theinput device250 may include, for example, atouch panel252, a (digital)pen sensor254, a key256, or anultrasonic input device258. Thetouch panel252 may use at least one of, for example, a capacitive type, a resistive type, an infrared type, and an ultrasonic type. Also, thetouch panel252 may further include a control circuit. Thetouch panel252 may further include a tactile layer and provide a tactile reaction to the user.
• The (digital)pen sensor254 may include, for example, a recognition sheet which is a part of the touch panel or is separated from the touch panel. The key256 may include, for example, a physical button, an optical key or a keypad. Theultrasonic input device258 may detect ultrasonic waves generated by an input tool through a microphone (for example, a microphone288) and identify data corresponding to the detected ultrasonic waves.
• The display260 (for example, the display170) may include apanel262, ahologram device264 or aprojector266. Thepanel262 may include a configuration identical or similar to that of thedisplay170 illustrated inFIG. 1. Thepanel262 may be implemented to be, for example, flexible, transparent, or wearable. Thepanel262 and thetouch panel252 may be implemented as one module. Thehologram264 may show a three dimensional image in the air using an interference of light. Theprojector266 may display an image by projecting light onto a screen. The screen may be located, for example, inside or outside theelectronic device201. According to an example, thedisplay260 may further include a control circuit for controlling thepanel262, thehologram device264, or theprojector266.
• Theinterface270 may include, for example, a High-Definition Multimedia Interface (HDMI)272, a Universal Serial Bus (USB)274, anoptical interface276, or a D-subminiature (D-sub)278. Theinterface270 may be included in, for example, thecommunication interface180 illustrated inFIG. 1. Additionally or alternatively, theinterface270 may include, for example, a Mobile High-definition Link (MHL) interface, a Secure Digital (SD) card/Multi-Media Card (MMC) interface, or an Infrared Data Association (IrDA) standard interface.
• Theaudio module280 bi-directionally converts a sound and an electronic signal. At least some components of theaudio module280 may be included in, for example, the input/output interface160 illustrated inFIG. 1. Theaudio module280 processes sound information input or output through, for example, aspeaker282, areceiver284, anearphone286, themicrophone288 or the like.
• Thecamera module291 is a device which can photograph a still image and a video. According to an example, thecamera module291 may include one or more image sensors (for example, a front sensor or a back sensor), an Image Signal Processor (ISP) (not shown) or a flash (for example, an LED or xenon lamp).
• Thepower managing module295 manages power of theelectronic device200. Although not illustrated, thepower managing module295 may include, for example, a Power Management Integrated Circuit (PMIC), a charger Integrated Circuit (IC), or a battery or fuel gauge.
• The PMIC may be mounted to, for example, an integrated circuit or an SoC semiconductor. A charging method may, for example, include wired and wireless methods. The charger IC charges a battery and prevent over voltage or over current from flowing from a charger. According to an example, the charger IC includes a charger IC for at least one of the wired charging method and the wireless charging method. The wireless charging method may include, for example, a magnetic resonance method, a magnetic induction method and an electromagnetic wave method, and additional circuits for wireless charging, for example, circuits such as a coil loop, a resonant circuit, a rectifier or the like may be added.
• The battery gauge measures, for example, a remaining quantity of thebattery296, or a voltage, a current, or a temperature during charging. Thebattery296 may store or generate electricity and supply power to theelectronic device200 by using the stored or generated electricity. Thebattery296 may include a rechargeable battery or a solar battery.
• The indicator297 shows particular statuses of theelectronic device200 or a part (for example, AP210) of theelectronic device200, for example, a booting status, a message status, a charging status and the like. Themotor298 converts an electrical signal to a mechanical vibration.
• Although not illustrated, theelectronic device200 may include a processing unit in the form of a processor (for example, GPU) for supporting a module TV. The processing unit for supporting the mobile TV may process, for example, media data according to a standard of Digital Multimedia Broadcasting (DMB), Digital Video Broadcasting (DVB), media flow or the like.
• Each of the components of the electronic device according to various examples of the disclosure may be implemented by one or more components and the name of the corresponding component may vary depending on a type of the electronic device. The electronic device according to various examples of the disclosure may include at least one of the above described components, a few of the components may be omitted, or additional components may be further included. Also, some of the components of the electronic device according to various examples of the disclosure may be combined to form a single entity, and thus may equivalently execute functions of the corresponding components before being combined.
• FIG. 3 is a block diagram illustrating an example program module. According to an example, a program module310 (for example, the program150) may include an Operating System (OS) for controlling resources related to the electronic device (for example, the electronic device101) and/or various applications (for example, application programs157) executed in the operating system. The operating system may be, for example, Android, iOS, Windows, Symbian, Tizen, Bada, or the like.
• Theprogram module310 may include akernel320, middleware330, an Application Programming Interface (API)360, and/or anapplication370. At least some of theprogram module310 may be preloaded on the electronic device, or may be downloaded from an external electronic device (e.g., theelectronic device102 or104, or the server106).
• The kernel320 (e.g., the kernel151) may include, for example, asystem resource manager321 and/or adevice driver323. Thesystem resource manager321 may perform the control, allocation, retrieval, or the like of system resources. According to an example of the disclosure, thesystem resource manager321 may include a process manager, a memory manager, a file system manager, or the like. Thedevice driver323 may include, for example, a display driver, a camera driver, a Bluetooth driver, a shared memory driver, a USB driver, a keypad driver, a Wi-Fi driver, an audio driver, or an Inter-Process Communication (IPC) driver.
• The middleware330 may, for example, provide a function required by theapplications370 in common or provide various functions to theapplications370 through theAPI360 so that theapplications370 can efficiently use limited system resources within the electronic device. According to an example, the middleware330 (for example, the middleware153) may include, for example, at least one of aruntime library335, an application manager341, awindow manager342, amultimedia manager343, aresource manager344, a power manager345, adatabase manager346, apackage manager347, aconnectivity manager348, anotification manager349, alocation manager350, agraphic manager351, and asecurity manager352.
• Theruntime library335 may include a library module that a compiler uses in order to add a new function through a programming language while theapplications370 are being executed. Theruntime library335 may perform input/output management, memory management, the functionality for an arithmetic function, or the like.
• The application manager341 may manage, for example, the life cycle of at least one of theapplications370. Thewindow manager342 may manage Graphical User Interface (GUI) resources used for the screen. Themultimedia manager343 may determine a format required to reproduce various media files, and may encode or decode a media file by using a coder/decoder (codec) appropriate for the corresponding format. Theresource manager344 may manage resources, such as a source code, a memory, a storage space, and the like of at least one of theapplications370.
• The power manager345 may operate together with a Basic Input/Output System (BIOS) to manage a battery or power and may provide power information required for the operation of the electronic device. Thedatabase manager346 may generate, search for, and/or change a database to be used by at least one of theapplications370. Thepackage manager347 may manage the installation or update of an application distributed in the form of a package file.
• Theconnectivity manager348 may manage a wireless connection such as, for example, Wi-Fi or Bluetooth. Thenotification manager349 may display or notify of an event, such as an arrival message, an appointment, a proximity notification, and the like, in such a manner as not to disturb the user. Thelocation manager350 may manage location information of the electronic device. Thegraphic manager351 may manage a graphic effect, which is to be provided to the user, or a user interface related to the graphic effect. Thesecurity manager352 may provide various security functions required for system security, user authentication, and the like. According to an example, when the electronic device (e.g., the electronic device101) has a telephone call function, the middleware330 may further include a telephony manager for managing a voice call function or a video call function of the electronic device.
• The middleware330 may include a middleware module that forms a combination of various functions of the above-described elements. The middleware330 may provide a module specialized for each type of OS in order to provide a differentiated function. Also, the middleware330 may dynamically delete some of the existing elements, or may add new elements.
• The API360 (e.g., the API155) may, for example, be a set of API programming functions, and may be provided with a different configuration according to an OS. For example, in the case of Android or iOS, one API set may be provided for each platform. In the case of Tizen, for example, two or more API sets may be provided for each platform.
• The applications370 (for example, the application programs157) may include, for example, one or more applications which can provide functions such ashome371,dialer372, SMS/MMS373, Instant Message (IM)374,browser375,camera376,alarm377,contacts378, voice dialer379,email380,calendar381,media player382,album383,clock384, health care (for example, measure exercise quantity or blood sugar), or environment information (for example, atmospheric pressure, humidity, or temperature information).
• According to an example, theapplications370 may include an application (hereinafter, referred to as an “information exchange application” for convenience of description) supporting information exchange between the electronic device (e.g., the electronic device101) and an external electronic device (e.g., theelectronic device102 or104). The application associated with information exchange may include, for example, a notification relay application for forwarding specific information to an external electronic device, or a device management application for managing an external electronic device.
• For example, the notification relay application may include a function of delivering, to the external electronic device (e.g., theelectronic device102 or104), notification information generated by other applications (e.g., an SMS/MMS application, an email application, a health care application, an environmental information application, etc.) of theelectronic device101. Further, the notification relay application may receive notification information from, for example, an external electronic device and provide the received notification information to a user.
• For example, the device management application may manage (for example, install, delete, or update) at least one function of an external electronic device (for example, theelectronic devices102 and104) communicating with the electronic device (for example, a function of turning on/off the external electronic device itself (or some components) or a function of adjusting luminance (or a resolution) of the display), applications operating in the external electronic device, or services provided by the external electronic device (for example, a call service and a message service).
• According to an example, theapplications370 may include applications (for example, a health care application of a mobile medical appliance or the like) designated according to attributes of the externalelectronic device102 or104. According to an example, theapplication370 may include an application received from the external electronic device (e.g., theserver106, or theelectronic device102 or104). According to an example, theapplication370 may include a preloaded application or a third party application which can be downloaded from the server. Names of the elements of theprogram module310, according to the above-described examples, may change depending on the type of OS.
• According to various examples, at least some of theprogram module310 may be implemented in software, firmware, hardware, or a combination of two or more thereof. At least some of theprogram module310 may be implemented (e.g., executed) by, for example, the processor (e.g., the processor210). At least some of theprogram module310 may include, for example, a module, a program, a routine, a set of instructions, and/or a process for performing one or more functions.
• The term “module” as used herein may, for example, mean a unit including one of hardware, software, and firmware or a combination of two or more of them. The “module” may be interchangeably used with, for example, the term “unit”, “logic”, “logical block”, “component”, or “circuit”. The “module” may be a minimum unit of an integrated component element or a part thereof. The “module” may be a minimum unit for performing one or more functions or a part thereof. The “module” may be mechanically or electronically implemented. For example, the “module” according to the disclosure may include at least one of an Application-Specific Integrated Circuit (ASIC) chip, a Field-Programmable Gate Arrays (FPGA), and a programmable-logic device for performing operations which have been known or are to be developed hereinafter.
• According to various examples, at least some of the devices (for example, modules or functions thereof) or the method (for example, operations) according to the disclosure may be implemented by a command stored in a computer-readable storage medium in a programming module form. The instruction, when executed by a processor (e.g., the processor130), may cause the one or more processors to execute the function corresponding to the instruction. The computer-readable storage medium may be, for example, thememory140.
• The computer readable recoding medium may include a hard disk, a floppy disk, magnetic media (for example, a magnetic tape), optical media (for example, a Compact Disc Read Only Memory (CD-ROM) and a Digital Versatile Disc (DVD)), magneto-optical media (for example, a floptical disk), a hardware device (for example, a Read Only Memory (ROM), a Random Access Memory (RAM), a flash memory), and the like. In addition, the program instructions may include high class language codes, which can be executed in a computer by using an interpreter, as well as machine codes made by a compiler. The aforementioned hardware device according to various examples may be configured to be operated as one or more software modules in order to perform the operation of the disclosure, and vice versa.
• The module or the programming module, e.g., processing circuitry, according to various examples may include one or more of the aforementioned components or may further include other additional components, or some of the aforementioned components may be omitted. Operations executed by a module, a programming module, or other component elements according to various examples may be executed sequentially, in parallel, repeatedly, or in a heuristic manner. Further, some operations may be executed according to another order or may be omitted, or other operations may be added. The examples disclosed herein are suggested for describing and helping understanding of disclosed technical contents, and do not limit the scope of the technology described in the disclosure. Accordingly, the scope of the disclosure should be construed as including all modifications or various other examples based on the technical idea of the disclosure. Hereinafter, information related to an apparatus and a method of providing information by using an object corresponding to an external electronic device (for example, theelectronic devices102 and104, or theserver106 are provided with reference toFIGS. 4 to 12.
• Hereinafter, an actual use environment and a virtual environment corresponding to the actual use environment will be described with reference toFIGS. 4 and 5.FIG. 4 illustrates an example of an actual use environment400, in which an electronic device (for example, the electronic device101) is linked with an external electronic device (for example, theelectronic devices102 and104, and the server106) within a designated space, andFIG. 5 illustrates an example of avirtual environment500 provided by an electronic device (for example, the electronic device101).
• Referring toFIG. 4, an actual use environment400 may be an environment, in which a plurality of electronic devices located within a space is linked with each other to provide a user410 with information, based on the space in which the user410 is located. For example, if the user410 is located in, for example, a living room, the actual use environment400 may be an environment, in which a plurality of electronic devices401 (413 to445) located in the living room provides the user410 with information. For another example, although not illustrated, if the user410 is located in an office, the actual use environment400 may be an environment providing the user410 with information through the linkage of a plurality of electronic devices located in the office.
• Referring toFIG. 4, the user410 may wear a Head-Mounted Device (HMD)401 (for example, the electronic device101) in a living room and look at a partial area490 of the living room. For example, the external electronic devices413 to445 for the HMD401 may be located in the living room. The external electronic devices413 to445 may be, for example, an electronic watch413, an electronic tattoo415, a lighting420, speakers423-1 and423-2, a television425, an interphone427, an air conditioner429, an electronic drawer430, a smart blind433, a smart phone440, a tablet PC443, or a laptop PC445. According to various examples, the external electronic devices413 to445 are not limited to the aforementioned devices, and may include various electronic devices. According to various examples, the electronic device401 is not limited to the illustrated HMD401, and may be various devices capable of providing the user410 with visual information.
• According to an example, at least some of the electronic device401 and the external electronic devices413 to445 may be connected with each other through wired or wireless communication. For example, at least some of the electronic device401 and the external electronic devices413 to445 may be directly connected through a local network including a home network or Device to Device (D2D) communication.
• According to an example, at least some of the external electronic devices413 to445 may output information (for example, a notification or contents). For example, the television425 may output avideo465. The tablet PC443 may output text483. The laptop PC445 may output a document485. Further, the electronic drawer430 may output a notification notifying that a drawer is opened. The smart phone440 may output a notification notifying a reception of a text message. For example, the notification may be a notification sound, vibration, scent, light, waves corresponding to a brainwave of a person, or an image, and the external electronic devices413 to445 may output at least one of the aforementioned notifications through which the user is capable of recognizing states or locations of the external electronic devices413 to445.
• According to an example, based on information, which has been output or is to be output through the external electronic devices413 to445, the electronic device401 may provide the user410 with contents corresponding to the information together with a virtual environment. For example, the notification provided by the electronic drawer430 and notifying that the drawer is opened may be provided in a form of an image through the electronic device401. For example, although not illustrated, the image may be displayed at least a part of a region, in which an object corresponding to the electronic drawer430 displayed on the virtual environment is displayed.
• Referring toFIG. 5, according to an example, avirtual environment500 may be provided through an electronic device501 (for example, a display (for example, the display170) functionally connected with the electronic device501). Thevirtual environment500 may include an image (for example, a still image or a video) identical or similar to an actual use environment (for example, the actual use environment400), in which theelectronic device501 or external electronic devices for theelectronic device501 are disposed. For example, if the actual use environment is a living room, thevirtual environment500 may include an image (an image of a picture of the living room or a drawing image of the living room) for the living room. For another example, thevirtual environment500 may include an image for an environment different from the actual use environment. For example, a space, in which theelectronic device501 and auser510 are actually located, is the “living room”, but thevirtual environment500 may include a “classroom”.
• According to an example, theelectronic device501 may provide thevirtual environment500 different from the actual space based on contents which have been provided (or are to be provided) to theuser510 or a user's input. For example, if theuser510 desires a simulation for “shopping”, theelectronic device501 may provide an image for a shopping mall as an example of thevirtual environment500. As another example, if the contents are movie contents, theelectronic device501 may provide an image of a movie theater as thevirtual environment500. According to an example, thevirtual environment500 may include a 3D image, as well as a 2D image.
• According to an example, one ormore objects513 to540 (hereinafter, candidate objects for convenience of description) corresponding to one or more external electronic devices (hereinafter, for convenience of description, “candidate electronic devices”) among the external electronic devices413 to445 disposed in the space (for example, the living room), in which theelectronic device501 is located, or other external electronic devices disposed in a space (not shown) different from the space may be displayed on thevirtual environment500.
• For example, the electronic watch413, the electronic tattoo415, the speakers423-1 and423-2, the television425, the air conditioner429, the smart phone440, and the smart blind433, which are disposed in a region490 (viewed by the user) corresponding to the gaze of theuser510 among the external electronic devices413 to445 may be selected as candidate electronic devices. Further, for example, a washing machine located at another space (for example, a utility room) connected with theelectronic device501 through a local network may be selected as a candidate electronic device. Additional information about the method of selecting the candidate electronic device will be described with reference toFIG. 6.
• The candidate objects513 to547 may be, for example, an image, text, a video, or an icon corresponding to the candidate electronic device. According to an example, the candidate objects513 to547 may be images related to an appearance of the candidate electronic device. For example, the candidate objects513 to547 may include speaker images523-1 and523-2 identical or similar to appearances of the speakers423-1 and423-2, atelevision image525 identical or similar to an appearance of the television425, animage529 different from an actual image of the air conditioner429 but representing the air conditioner429, animage533 different from an actual image of the smart blind433 but representing the smart blind433 and animage540 different from an actual image of the smart phone440 but representing the smart phone440, e.g., icons.
• According to an example, the candidate objects515 to547 may include, for example, text (for example, a product name or identification information) for the candidate electronic device. For example, the candidate objects515 to547 may be anoutput window515 including “electronic tattoo” that is text corresponding to the electronic tattoo415.
• According to an example, the candidate objects513 to547 may be icons for the candidate electronic devices. For example, the candidate objects513 to547 may include anelectronic watch icon513 representing the electronic watch413 or awashing machine icon547 representing a washing machine.
• According to an example, the candidate objects513 to547 may be displayed on a region of thevirtual environment500 corresponding to relative positions of the candidate electronic devices for theelectronic device501. Theuser510 may use theHMD501 that is theelectronic device510 at a first location corresponding to a left-rear side of the living room. Theuser510 may be attached with the electronic tattoo415 on his/her arm, and hold the smart phone440 with a hand. Accordingly, locations of the electronic tattoo415 and the smart phone440 may be the same as or similar to the location of theHMD501. For example, theelectronic tattoo object515 corresponding to the electronic tattoo415 and thesmart phone object540 corresponding to the smart phone440 may be displayed in a first region corresponding to a left lower side of thevirtual environment500 as candidate objects.
• For example, the television425 may be located at a second location corresponding to a center front side of the living room. In this case, thetelevision object525 corresponding to the television425 may be displayed in a second region corresponding to the center front side of thevirtual environment500 as a candidate object. For example, the first speaker423-1 and the second speaker423-2 may be located at third locations which are both sides of the television425 in the living room. In this case, the first speaker object523-1 and the second speaker object523-2 corresponding to the first speaker423-1 and the second speaker423-2, respectively, may be displayed on third regions, which are both sides of thetelevision525 in thevirtual environment500, as a candidate objects.
• For example, the air conditioner479 may be located at a fourth location corresponding to a front side of the living room. For example, theair conditioner object529 corresponding to the air conditioner429 may be displayed in a fourth region corresponding to a right center side of thevirtual environment500 as a candidate object. For example, the smart blind433 may be located at a fifth location corresponding to a right front side of the air conditioner579 in the living room. In this case, the smartblind object533 corresponding to the smart blinder433 may be displayed in a fifth region corresponding to a right upper end of theair conditioner object529 in thevirtual environment500 as a candidate object.
• According to an example, the candidate objects513 to547 may be displayed on aregion503 designated in thevirtual environment500. For example, the rightlower region503 may be designated as a region for displaying the candidate objects513 to547 in thevirtual environment500. Accordingly, theelectronic watch object513 corresponding to the electronic watch413 and thewashing machine547 corresponding to the washing machine may be displayed on theregion503 designated for displaying the objects.
• According to an example, contents may be provided in relation to the region, on which at least some objects (hereinafter, target objects for convenience of description) among one or more candidate objects displayed in thevirtual environment500. The contents may correspond to information to be output by at least some electronic devices (hereinafter, referred to as target electronic devices for convenience of description) corresponding to the target objects among one or more candidate electronic devices.
• For example,text565 “Body temperature: 37°, and Stress: 90%” corresponding to information on a body temperature and a degree of stress of theuser510, which is provided through the electronic tattoo415, may be provided through the region on which theelectronic tattoo object515 is displayed. For example, contents465 (for example, a movie) to be output through the television425 may be provided through the region, on which thetelevision object525 that is the target object is displayed. For example, thetext message icon590 corresponding to notification information about the reception of the text message to be provided by the smart phone440 may be provided through the region on which thesmart phone object540 that is the target object is displayed. For example,text597 corresponding to notification information about “washing is completed” to be output by the washing machine may be provided through an upper region of the region, on which thewashing machine object547 that is the target object is displayed.
• According to an example, a control menu (for example, an image, text, or an icon) for controlling the target electronic device may be provided in relation to the region, on which the target object is displayed, in thevirtual environment500. For example, acontrol menu583 for opening or closing the smart blind433 may be provided through the region, on which the smartblind object533 is displayed.
• FIG. 6 is a block diagram illustrating an example information providing module (for example, theinformation providing module110 in the form, for example, of processing circuitry) included in an electronic device (for example, the electronic device101). Hereinafter, the description of the same or similar parts as those inFIGS. 1 to 5 will be omitted. Referring toFIG. 6, aninformation providing module610 may include, for example, adevice checking module620, anobject display module630, aninformation obtaining module640, acontent providing module650, and adevice control module660. According to an example, at least some of thedevice checking module620, theobject display module630, theinformation obtaining module640, thecontent providing module650, and thedevice control module660 may be implemented by hardware, software, or firmware, or any combination thereof which is capable of performing or being configured to perform designated operations.
• Thedevice checking module620 may be configured to check, for example, a candidate electronic device among one or more external electronic devices (for example, theelectronic devices102 and104, and the server106) for the electronic device. According to an example, thedevice checking module620 may be configured to recognize (for example, scan) one or more external electronic devices communicable with the electronic device. One or more external electronic devices may be devices connected with the electronic device through a network (for example, the network162) or a short range communication (for example, the short range communication164).
• For example, thedevice checking module620 may be configured to transmit a request signal requesting communication to the plurality of external electronic devices through Wi-Fi communication (or Bluetooth, infrared communication, or short range communication). Thedevice checking module620 may be configured to receive a response signal for the request signal from one or more external electronic devices communicable through the Wi-Fi communication among the plurality of external electronic devices. Accordingly, thedevice checking module620 may be configured to recognize the one or more external electronic devices. For example, thedevice checking module620 may be configured to receive a signal transmitted (for example, broadcasted) from one or more external electronic devices (a Bluetooth Low Energy (BLE) tag included in the one or more external electronic devices) without requesting communication from the plurality of external electronic devices. Accordingly, thedevice checking module620 may recognize the one or more external electronic devices.
• According to an example, thedevice checking module620 may be configured to recognize one or more external electronic devices in response to the use (e.g., wearing) of the electronic device by a user. For example, if the electronic device is a wearable device, thedevice checking module620 may be configured to check whether the electronic device is worn by the user. Accordingly, if the electronic device is worn by the user, thedevice checking module620 may recognize, for example, the one or more external electronic devices. On the other hand, if the electronic device is not worn by the user, thedevice checking module620 may not recognize, for example, the one or more external electronic devices. According to various examples, thedevice checking module620 may recognize one or more external electronic devices based on various user's inputs (for example, selection of a menu item related to the recognition of the external electronic device), as well as whether the user uses the electronic device.
• According to various examples, thedevice checking module620 may be configured to automatically recognize one or more external electronic devices. For example, thedevice checking module620 may be configured to periodically (for example, once for about each one hour) recognize one or more external electronic devices. For example, thedevice checking module620 may recognize one or more external electronic devices in response to an operation of the one or more external electronic devices. For example, thedevice checking module620 may detect a signal corresponding to a turn-on operation of the one or more external electronic devices. Thedevice checking module620 may recognize the one or more external electronic devices based on the signal.
• According to an example, thedevice checking module620 may be configured to select a candidate electronic device from the recognized one or more external electronic devices. According to an example, thedevice checking module620 may be configured to select an electronic device located within a designated range from the electronic device as a candidate electronic device. The designated range may include at least one of, for example, a range detectable by a sensor (for example, the image sensor) included in the electronic device, a range, in which short range communication (for example, the short range communication164) with the electronic device may be established, a range, in which communication with the electronic device with a designated signal intensity or more may be established, or ranges connected with each other through a local network.
• For example, thedevice checking module620 may select an electronic device located at a place, which the user may visually recognize, among the recognized one or more external electronic devices as a candidate electronic device. To this end, thedevice checking module620 may be configured to obtain an image at a location corresponding to the gaze of the user through the image sensor included in the electronic device. Thedevice checking module620 may select one or more electronic devices include in the image as a candidate electronic device.
• For example, thedevice checking module620 may be configured to select an electronic device located to be approximate to the user among the recognized one or more external electronic devices as a candidate electronic device. To this end, thedevice checking module620 may select one or more electronic devices, which may establish short range communication (for example, Wi-Fi, Bluetooth, or infrared communication) with the electronic device among the one or more external electronic devices connected with the electronic device through various communication methods, as a candidate electronic device.
• For example, the electronic device may include an infrared communication module communicable with an external electronic device corresponding to a designated direction. Thedevice checking module620 may select an external electronic device located in a desired direction as a candidate electronic device by using the infrared communication module. For example, thedevice checking module620 may receive an infrared signal from one or more external electronic devices located in a direction corresponding to the infrared communication module through infrared communication. For example, thedevice checking module620 may select the one or more external electronic devices as a candidate electronic device.
• Further, thedevice checking module620 may be configured to select, for example, one or more electronic devices, in which a communication signal intensity between the electronic device and one or more external electronic devices is a designated signal intensity or more (for example, high when signal intensity is divided into high, middle, and low), among the recognized one or more external electronic devices as a candidate electronic device.
• Further, for example, the recognized one or more external electronic devices may be connected within the electronic device through various networks. For example, thedevice checking module620 may select one or more electronic devices connected through the local network (for example, a home network) among the one or more external electronic devices as a candidate electronic device.
• According to an example, thedevice checking module620 may be configured to select a candidate electronic device based on an input, such as, for example, a user input. For example, thedevice checking module620 may be configured to recognize, for example, the gaze of the user for the electronic device through an eye recognizing sensor functionally connected with the electronic device (for example, included in the electronic device). Thedevice checking module620 may be configured to select an electronic device corresponding to the gaze of the user among the one or more external electronic devices as a candidate electronic device. Further, for example, thedevice checking module620 may be configured to recognize, for example, a motion of the user (for example, a motion of indicating the electronic device with a hand or a finger of the user) through the image sensor functionally connected with the electronic device. Thedevice checking module620 may select an electronic device corresponding to the motion (for example, the electronic device indicated by the hand or the finger of the user) as a candidate electronic device.
• According to an example, thedevice checking module620 may be configured to select a candidate electronic device based on operation states of one or more communicable external electronic devices. For example, thedevice checking module620 may be configured to select an electronic device which is outputting (or is to output) information (for example, contents, a sound, scent, light, radio waves, or vibrations) among one or more external electronic devices as a candidate electronic device. For example, thedevice checking module620 may be configured to obtain, for example, operation information about an external electronic device regarding whether one or more external electronic devices are outputting information, through wired or wireless communication. Based on the operation information, thedevice checking module620 may select the electronic device, which is outputting the information, as a candidate electronic device.
• For example, thedevice checking module620 may be configured to obtain, for example, an image of a first external electronic device, which outputs a video through a display and an image of a second external electronic device, which does not output the information, through the image sensor. For example, based on the images, thedevice checking module620 may select the first external electronic device as a candidate electronic device.
• Theobject display module630 may be configured to display a candidate object corresponding to the candidate electronic device. According to an example, theobject display module630 may be configured to display a virtual environment (for example, the virtual environment500) for displaying a candidate object through a display (for example, the display170) functionally connected with the electronic device. According to an example, theobject display module630 may be configured to select a virtual environment to be displayed through the electronic device among a plurality of virtual environments based on at least one input for the electronic device. Theobject display module630 may be configured to provide the selected virtual environment. According to an example, theobject display module630 may be configured to display a virtual environment obtained through the candidate electronic device or another external electronic device through the display. For example, theobject display module630 may be configured to obtain a virtual environment through a candidate electronic device based on a communication connection between the candidate electronic device and the electronic device or a request of the user. Additional information on the method of providing the virtual environment will be described with reference toFIG. 7 below.
• According to an example, theobject display module630 may be configured to display the candidate object within the virtual environment. For example, theobject display module630 may display the candidate object through a region corresponding to a location of the candidate electronic device in an actual use environment or a region (for example, the region503) designated for displaying the candidate object.
• According to an example, theobject display module630 may be configured to differently display the candidate objects based on processing information for the candidate object or the virtual environment. The processing information may, for example, include information on a time, for which the gaze of the user for the candidate electronic device stays, or a degree of communication connection (paring) with the electronic device. According to an example, theobject display module630 may be configured to display the candidate object with a chroma, luminance, brightness, light and shade, transparency, color, shape, shaking, motion or a size designated according to the time, for which the gaze of the user for the candidate electronic device stays, or a communication connection state. For example, if the user views the candidate electronic device for a first time (for example, about 5 seconds), theobject display module630 may be configured to display the candidate object with a first color (for example, black and white colors), if the user views the candidate electronic device for a second time (for example, about 30 seconds) longer than the first time, theobject display module630 may display the candidate object with a second color (for example, colors).
• For example, theobject display module630 may be configured to display the candidate object with a different size according to the communication connection state between the electronic device and the candidate electronic device. For example, the communication connection state may be divided into a “communication connection attempt”, “ongoing communication connection”, and “communication connection completion”. If the communication connection state between the electronic device and the candidate electronic device is the “communication connection attempt”, theobject display module630 may be configured to display the candidate object with a first size. If the communication connection state between the electronic device and the candidate electronic device is changed to the “ongoing communication connection”, theobject display module630 may change a size of the candidate object to a second size larger than the first size. If the communication connection state between the electronic device and the candidate electronic device is changed to the “communication connection completion”, theobject display module630 may change a size of the candidate object to a third size larger than the second size.
• For convenience of description, the communication connection state is divided into three steps (for example, the “communication connection attempt”, the “ongoing communication connection”, and the “communication connection completion”), but may be variously set according to various examples. For example, the communication connection state may be divided into four steps, that is, “communication connection attempt”, “ongoing communication connection by 40%”, “ongoing communication connection by 80%”, and “communication connection completion”. Additional information about the method of displaying the candidate object with a designated format will be described with reference toFIG. 11.
• According to an example, theobject display module630 may be configured to display a user interface (for example, a progress bar or a loading bar) indicating a degree of processing the candidate object or the virtual environment in relation to the candidate object. For example, theobject display module630 may be configured to display a progress bar for displaying a time, for which the gaze of the user for the candidate object stays, in the region, in which the candidate object is displayed. If the gaze of the user for the candidate object stays for the first time (for example, about 5 seconds), theobject display module630 may display a first progress bar indicating a first percentage (for example, about 30%). Further, If the gaze of the user for the candidate object stays for the second time (for example, about 15 seconds) longer than the first time, theobject display module630 may display a second progress bar indicating a second percentage (for example, about 90%) larger than the first percentage.
• According to an example, theobject display module630 may be configured to differently display the plurality of candidate objects so that the plurality of objects is discriminated from each other. Theobject display module630 may be configured to display, for example, the plurality of candidate objects with different frequencies, colors, or brightness so that the plurality of candidate objects is discriminated through a brainwave signal of the user viewing the plurality of candidate objects. For example, theobject display module630 may be configured to check a first region of the display for displaying a first candidate object and a second region of the display for displaying a second candidate object. Theobject display module630 may be configured to display the first candidate object at a first frequency (for example, about 60.2 Hz) through the first region, and display the second candidate object at a second frequency (for example, about 60.4 Hz) through the second region.
• For example, theobject display module630 may be configured to display the first candidate object with a first color (for example, red) and display the second candidate object with a second color (for example, blue). Further, theobject display module630 may be configured to display the first candidate object with first brightness (for example, about 2 lux), and display the second candidate object with second brightness (for example, about 5 lux).
• Theinformation obtaining module640 may be configured to obtain, for example, information (for example, thevideo465, the text483, and the text message485) to be output (or currently output) through a target electronic device corresponding to a target object among one or more candidate objects. According to an example, theinformation obtaining module640 may be configured to determine a target object based on a user's input (for example, a gesture, a touch, or the eye), execution information of the candidate electronic device, or the motion information about the candidate electronic device.
• For example, theinformation obtaining module640 may be configured to check the gaze of the user based on a user's input. If the gaze of the user corresponds to the first candidate object, theinformation obtaining module640 may be configured to determine the first candidate object as a target object. If the gaze of the user corresponds to a second candidate object, theinformation obtaining module640 may determine the second candidate object as a target object.
• For example, if the electronic device is an HMD, theinformation obtaining module640 may be configured to recognize, for example, a motion (for example, rotation) of a head of the user based on a user's input through a motion sensor functionally connected with the electronic device (for example, included in the electronic device). If the motion corresponds to a first motion corresponding to the first candidate object (for example, the head rotates in a first direction corresponding to the first candidate object), theinformation obtaining module640 may determine the first candidate object as a target object. If the motion corresponds to a second motion corresponding to the second candidate object (for example, the head rotates in a second direction corresponding to the second candidate object), theinformation obtaining module640 may determine the second candidate object as a target object. For example, the electronic device (for example, the electronic device101) may use at least one of thegesture sensor240A, thegyro sensor240B, and theacceleration sensor240E in thesensor module240 as a motion sensor.
• For example, theinformation obtaining module640 may be configured to determine a target object based on biometric information (for example, a brainwave) of the user as a user's input. For example, the brainwave of the user may be differently measured according to visual information viewed by the user (for example, an output frequency, a color, or brightness of visual information). For example, if a first image is output on the display at the first frequency (for example, about 60.2 Hz), the brainwave of the user viewing the first image may be a first signal. Further, if a second image is output on the display at the second frequency (for example, about 60.4 Hz), the brainwave of the user viewing the second image may be a second signal. Accordingly, theinformation obtaining module640 may be configured to check an image viewed by the user corresponding to the brainwave between the first image and the second image by using the brainwave obtained through a brainwave recognizing sensor functionally connected with the electronic device.
• For example, theobject display module630 may be configured to output the first candidate object through the display at the first frequency, and output the second candidate object through the display at the second frequency. In this case, theinformation obtaining module640 may be configured to obtain the brainwave of the user. If the brainwave of the user is the first signal corresponding to the first frequency, theinformation obtaining module640 may be configured to determine the first candidate object as a target object. If the brainwave of the user is the second signal corresponding to the second frequency, theinformation obtaining module640 may be configured to determine the second candidate object as a target object.
• For example, theinformation obtaining module640 may be configured to check information about reception of a notification by the candidate electronic device as execution information. Accordingly, theinformation obtaining module640 may be configured to determine the candidate object corresponding to the candidate electronic device as a target object. For example, theinformation obtaining module640 may be configured to check information about completion of washing by the washing machine that is the candidate electronic device as execution information. Accordingly, theinformation obtaining module640 may determine the washing machine object corresponding to the washing machine as a target object.
• Theinformation obtaining module640 may be configured to check, for example, motion information about one or more candidate electronic devices. For example, if the motion information corresponds to designated motion information, theobject display module630 may be configured to determine a candidate object corresponding to the candidate electronic device as a target object. For example, if a user wearing an electronic watch raises his/her arm in a direction of a face of the user, theinformation obtaining module640 may be configured to determine that the user intends to check the electronic clock. Accordingly, the motion information corresponding to the movement of the electronic watch in an upper direction may be set as the designated motion information. For example, a motion sensor included in the electronic watch may detect the motion information corresponding to the movement of the electronic watch in the upper direction. For example, theinformation obtaining module640 may obtain the motion information from the electronic watch. Theinformation obtaining module640 may determine the electronic watch as a target electronic device based on the motion information.
• According to an example, motion information about one or more candidate electronic devices may be checked by various sensors, such as the image sensor functionally connected with the one or more candidate electronic devices or the electronic device, as well as a motion sensor included in the one or more candidate electronic devices.
• According to an example, theinformation obtaining module640 may be configured to determine a group including one or more candidate objects a target object, as well as determine each candidate object as a target object. The group may be determined based on at least one of, for example, a user's input, a region, in which the candidate object is displayed, and functions of the candidate electronic devices.
• For example, based on a user's input (for example, a touch input), first to third candidate objects may be determined as a first group, and fourth and fifth candidate objects may be determined as a second group. For example, the first and second candidate objects located in the first region of the virtual environment may be determined as the first group, and the third to fifth candidate objects located in the second region of the virtual environment may be determined as the second group.
• For example, one or more candidate objects corresponding to one or more candidate electronic devices, in which a relevant function is executed, may be determined as one group. For example, a monitor object corresponding to a monitor for outputting a video and a speaker object corresponding to a speaker for outputting a sound corresponding to the video may be determined as one group. Further, a game console object corresponding to a game console for controlling a game, a monitor object corresponding to a monitor for displaying an image of the game, and a speaker object corresponding to a speaker for outputting a sound for the game may be determined as one group.
• According to an example, theinformation obtaining module640 may be configured to obtain information to be output through the target electronic device corresponding to the target object. For example, a movie may be currently output through the target electronic device. Theinformation obtaining module640 may be configured to obtain information on at least a part (a next part of a part output by the target electronic device) of the currently output movie. For example, when a notification (for example, a notification for reception of a text message when the target electronic device is a smart phone) is output (or to be output) by the target electronic device, theinformation obtaining module640 may receive information about the notification. For example, when a smart phone that is the target electronic device receives a text message, the smart phone may need to output at least one of a ring sound, a vibration, text message contents, and text message sender information corresponding to the reception of the text message. For example, theinformation obtaining module640 may be configured to obtain at least one of information indicating that the text message is received, text message contents, and the text message sender information.
• According to an example, theinformation obtaining module640 may be configured to directly obtain information from the target electronic device through wireless or wired communication (for example, D2D communication). Further, according to another example, theinformation obtaining module640 may be configured to obtain information about the target electronic device through another electronic device (for example, theelectronic devices102 and104, and the server106) connected with the target electronic device. For example, theinformation obtaining module640 may obtain the information through a server managing the information to be output by the target electronic device.
• Thecontent providing module650 may be configured to provide, for example, contents corresponding to the obtained information in relation to the region, on which the target object is displayed (for example, within the displayed region or a region closed to the displayed region). According to an example, thecontent providing module650 may be configured to determine contents corresponding to the obtained information. For example, when the obtained information is information about a notification, thecontent providing module650 may be configured to determine an image, a video, text, an icon, a sound, or a vibration related to the notification as the contents corresponding to the notification.
• According to an example, the obtained information may be information about contents (for example, a video, an image, or text) output by the target electronic device. For example, thecontent providing module650 may be configured to determine contents identical or similar to the contents output by the target electronic device or contents obtained by processing (for example, changing a format, resolution, or a size of data) at least a part of the contents output by the target electronic device as the contents corresponding to the information.
• For example, the obtained information may be information about a second part, which is a next part of a first part of a movie currently output through the target electronic device. For example, thecontent providing module650 may be configured to determine the second part of the movie as contents corresponding to the information. For example, a data format of the video that is the obtained information may be a first data format. For example, thecontent providing module650 may be configured to determine the video, of which the data format is changed from the first data format to the second data format, as contents corresponding to the information.
• According to an example, thecontent providing module650 may be configured to provide the contents to the region, on which the target object is displayed, the region closed to (or connected with) the region, on which the target object is displayed, or a direction corresponding to the region, on which the target object is displayed. For example, when the target object is an image of a television, thecontent providing module650 may be configured to display contents on a partial region (for example, an image part corresponding to a display of the television) included in the television image. For example, if the target object is an icon (for example, the washing machine icon547) for the target electronic device, thecontent providing module650 may be configured to display the contents (for example, the contents597) on an upper, lower, left, or right region of the icon. For example, thecontent providing module650 may be configured to display the contents together with visual information (for example, a connection line) connecting the target object and the contents. For example, thecontent providing module650 may display the contents on a region different from that of the target region, and then move the contents to the region, on which the target object is displayed.
• For example, the electronic device may include a first speaker located in a first direction (for example, a right side) of the display and a second speaker located in a second direction (for example, a left side) of the electronic device. A first target object may be displayed on a region corresponding to the first direction of the display. In this case, thecontent providing module650 may be configured to output a louder sound through the first speaker than the second speaker so that the user recognizes as if the sound that is the content is output in the first direction corresponding to the first target object. Otherwise, thecontent providing module650 may be configured to output the sound only through the first speaker.
• A second target object may be displayed on a region corresponding to the second direction of the display. For example, thecontent providing module650 may output a louder sound through the second speaker than the first speaker so that the user recognizes as if the sound that is the content is output in the second direction corresponding to the second target object. Otherwise, thecontent providing module650 may output the sound only through the second speaker.
• According to an example, thecontent providing module650 may be configured to provide another information obtained through an electronic device different from the target electronic device in relation to the region, on which the target object corresponding to the target electronic device is displayed. For example, information (for example, writing contents for a lecture when the content is a video lecture) related to contents provided to the region, on which the target object corresponding to the target electronic device is displayed, may be obtained through an electronic device different from the target electronic device. For example, thecontent providing module650 may be configured to provide the information to the region, on which the target object is displayed (or the region close to the region), in relation to the contents. A method of providing the obtained information through another electronic device will be described with reference toFIG. 10.
• According to an example, thecontent providing module650 may be configured to control (for example, temporarily stop or turn off) the provision of the contents based on external environment information (for example, external object recognition) for the electronic device. For example, thecontent providing module650 may be configured to check for, for example, a person located around the electronic device (for example, within about 1 m) as external environment information through the image sensor functionally connected with the electronic device while providing the contents. For example, thecontent providing module650 may be configured to stop providing the contents so as to enable the user of the electronic device to recognize that a person is located in a neighboring region. Otherwise, thecontent providing module650 may be configured to display information indicating that a person is located in a neighboring region. For example, thecontent providing module650 may display an image (for example, a picture of the person) or text corresponding to the person.
• Thedevice control module660 may be configured to control, for example, a function of the target electronic device corresponding to the target object. According to an example, thedevice control module660 may be configured to control an output function of the target electronic device. For example, the contents corresponding to the output (for example, currently output) information through the target electronic device through thecontent providing module650 may be output through the electronic device. Accordingly, the information may be provided through the target electronic device and the contents corresponding to the information may be simultaneously provided by the electronic device. It may be difficult for the user to simultaneously check the information output through the target electronic device and the contents provided by the electronic device.
• Accordingly, when the user uses the electronic device, thedevice control module660 may be configured to control the target electronic device so that the information is not output through the target electronic device. On the other hand, when the user does not use the electronic device, thedevice control module660 may be configured to control the target electronic device so that the information is output through the target electronic device. Further, thedevice control module660 may be configured to stop the output of the contents for the information by the electronic device.
• For example, if the electronic device is a wearable electronic device, thedevice control module660 may be configured to determine whether the electronic device is used based on the electronic device is worn by the user. For example, the contents may be provided by a television, which is the external electronic device of the electronic device. If the electronic device is an HMD, thedevice control module660 may check whether the HMD is worn by the user.
• If the HMD is worn by the user, thedevice control module660 may be configured to control the television so that the contents are not provided through the television (turn off only a screen of the television) or turn off the television. Then, If the HMD is detached from the user, thedevice control module660 may control the television or turn on the television so that the contents are output through the television again. The method of controlling the target electronic device will be described with reference toFIG. 8 below.
• According to an example, thedevice control module660 may also be configured to control (turn off) an operation of another external electronic device for the electronic device, as well as the target electronic device. For example, thedevice control module660 may be configured to turn off a lighting located around the electronic device and at least some functions of the television (turn off only a screen output function of the television) when the electronic device is used by the user. On the other hand, thedevice control module660 may be configured to turn on the lighting and the television again when the electronic device is not used by the user.
• According to an example, thedevice control module660 may be configured to provide a control menu for controlling the contents output through the target electronic device or the electronic device in relation to the target object. For example, when the target electronic device is the lighting, thedevice control module660 may be configured to display control menu items corresponding to a function of “turning off” and “turning on” the lighting on a region, on which the lighting object is displayed. When the user selects the control menu item corresponding to “turning off”, thedevice control module660 may be configured to control the lighting so that the lighting is turned off. The method of providing the control menu item will be described with reference toFIG. 8 below.
• According to an example, thedevice control module660 may be configured to transmit at least some of the contents output through the target object or the information (for example, some contents or link information about the contents) related to the contents to an electronic device corresponding to another target object. Accordingly, at least some of the information may be provided through the electronic device corresponding to another target object. For example, thedevice control module660 may be configured to select at least a part of an advertisement image that is the content provided through a region, on which a first target object is displayed.
• According to an example, thedevice control module660 may be configured to transmit link information related to an advertisement product corresponding to the advertisement image to a target electronic device corresponding to a second target object based on a user's input (for example, a drag from the first target object to the second target object or a movement of the gaze of the user from the first target object to the second target object) for a region, on which the second target object is displayed. The method of transmitting the contents or the information related to the contents will be described with reference toFIG. 9 below.
• FIG. 7 illustrates an example of a user interface (for example, the virtual environment500) provided through an electronic device (for example, the electronic device101). Referring toFIG. 7, a background of a living room, in which a user710 is located, may be displayed through a display703 of an electronic device701 as a background of a virtual environment705 (for example, the virtual environment500). According to an example, contents709, which may be output through a television disposed in the living room, may be displayed through a television object707 corresponding to the television. For example, the user710 may watch the contents709 through the television object707 within the virtual environment705 identically or similarly to the contents709 provided by the actual television in the actual living room.
• According to an example, a selection menu720, through which the background of the virtual environment705 may be changed, may be provided to the user710 through the virtual environment705. For example, when the television object707 displayed within the virtual environment705 is displayed with a first depth effect (or three-dimensional effect), the selection menu720 may be displayed with a second depth effect. For example, the selection menu720 having the second depth effect may be recognized to be closer to the user710 than the television object707 having the first depth effect. According to an example, the selection menu720 may be more transparently displayed than other objects within the virtual environment705.
• The selection menu720 may include, for example, a first menu item721 having a background of a lecture room, a second menu item723 having a background of a conference room, and a third menu item725 having a background of a café, or other menu items (not shown). According to an example, the first menu item721 to the third menu item725 may be moved in a specific direction729 (for example, a right direction) based on the gaze or a gesture of the user710. For example, inFIG. 7, the first menu item721, the second menu item723, and the third menu item725 are sequentially disposed in the right direction (for example, the specific direction729) from a left side, but the third menu item725, the first menu item721, and the second menu item723 may be sequentially disposed from the left side to the right side according to an input of the user710.
• According to an example, the second menu item723 may be selected by the gaze or the gesture of the user710. For example, the user710 may select the second menu item723 by viewing the second menu item723 for a predetermined time, or by touching the second menu item723. According to an example, the electronic device701 may provide the user710 with visual information notifying that the second menu item723 is selected. For example, the electronic device701 may display the second menu item723 to be larger than the first menu item721 or the third menu item725, display the second menu item723 to be stronger than the first menu item721 or the third menu item725, display the second menu item723 to be flickered, or display the second menu item723 with a shadow.
• FIG. 8 illustrates an example of a user interface (for example, the virtual environment500) provided through an electronic device (for example, the electronic device101). Descriptions of parts identical or similar to those ofFIG. 7 will be omitted, and reference numerals of parts identical or similar to those ofFIG. 7 will be denoted with the identical or similar reference numerals.
• Referring toFIG. 8, contents809 (for example, the contents709) providable through anactual television851 disposed in a laboratory room800, in which auser810 is located, may be provided through a television object807 (for example, the television object707) in a virtual environment805 (for example, the virtual environment705). For example, if theuser810 wears an electronic device801 (for example, the electronic device701), theelectronic device801 may display thecontents809 within at least a partial region of thevirtual environment805 displayed through theelectronic device801.
• According to an example, if the contents809 (for example, broadcasting or Electronic Program Guide (EPG)) are displayed through thetelevision object807, thetelevision851 may continuously output thecontents809, but may also stop providing thecontents809 and provide adefault image849. Thedefault image849 may include, for example, an image (for example, black data) having the smaller amount of data than that of thecontents809 or the smaller amount of current consumed than that consumed if thecontents809 are output. For example, if thecontents809 are displayed through thetelevision object807, at least some functions (for example, only a screen output function of the television) of thetelevision851 may be turned off, and some other functions of the television851 (an operation of waiting for reception of a signal provided by the electronic device101) may be performed. According to an example, if thecontents809 are displayed through thetelevision object807, thetelevision851 may be turned off.
• According to an example, acontrol menu830, through which thetelevision851 may be controlled, may be provided to theuser810 through thevirtual environment805. For example, thecontrol menu830 may include achannel menu837, through which a channel of thetelevision851 may be changed. Although not illustrated, thecontrol menu830 may include various menu items including volume, screen resolution, pause, play, fast forward, rewind, or fast play. For example, if thetelevision object807 displayed within thevirtual environment805 is displayed with a first depth effect (or three-dimensional effect), theselection menu830 may be displayed with a second depth effect or more transparently than other objects.
• Theuser810 may select a channel of thetelevision851 through ahardware key803 formed in theelectronic device801. Otherwise, theuser810 may select, for example, a channel of thetelevision851 by viewing or touching with a finger at least one of first tothird channel information831,833, and835 displayed within thevirtual environment805 for a predetermined time. According to an example, thefirst channel information831 to thethird channel information835 may be moved in a specific direction829 (for example, a left direction) based on the gaze or a gesture of theuser810.
• According to an example, the sequentially disposed first tothird channel information831,833, and835 may be displayed with different sizes based on the selectedsecond channel information833. For example, the selectedsecond channel information833 may be displayed to be largest, and the non-selectedfirst channel information831 andthird channel information835 may be displayed to be relatively smaller than thesecond channel information833. According to an example, other non-selected channel information may be displayed to be gradually smaller when being far from the selectedsecond channel information833.
• FIG. 9 illustrates an example of a user interface (for example, the virtual environment500) provided through an electronic device (for example, the electronic device101). Descriptions of parts identical or similar to those ofFIG. 7 or 8 will be omitted, and reference numerals of parts identical or similar to those ofFIG. 7 or 8 will be denoted with the identical or similar reference numerals.
• Referring toFIG. 9, contents909 (for example, the contents709) providable through amonitor951 or through amonitor object907 corresponding to anactual monitor951 may be provided in a virtual environment905 (for example, the virtual environment705) corresponding to an office900. According to an example, anicon951, through which information related to thecontents909 is transmittable to anactual notebook computer960, may be displayed in thevirtual environment905.
• Theicon951 may be displayed on, for example, at least a partial region of themonitor object907, on which thecontents909 are displayed. According to an example, theicon951 may be displayed in a form of an image or text representing a transportation means (for example, an automobile, a cart, a bicycle, a wagon, or a wheel). According to an example, if auser910 touches theicon951 or views theicon951 for a predetermined time, theicon951 may be activated. According to an example, if theicon951 is activated, theicon951 may be flickered, displayed thicker, displayed larger, or finely move. For example, if theicon951 is shaped like a cart, the cart may roll wheels in place or while moving forward in a specific direction929 (for example, a front direction).
• According to an example, if theicon951 is moved to anotebook computer object961 corresponding to thenotebook computer960 based on the gaze or the gesture of theuser910, the information related to thecontents909 may be transmitted to thenotebook computer960. The information related to thecontents909 may include, for example, at least some contents of thecontents909 currently displayed through themonitor object907, link information about thecontents909, and detailed information about an image, terms, or text included in thecontents909.
• FIG. 10 illustrates an example of a user interface (for example, the virtual environment500) provided through an electronic device (for example, the electronic device101). Descriptions of parts identical or similar to those ofFIGS. 7 to 9 will be omitted, and reference numerals of parts identical or similar to those ofFIGS. 7 to 9 will be denoted with the identical or similar reference numerals.
• Referring toFIG. 10, auser101 may use a first electronic device1001 (for example, the electronic device101), a secondelectronic device1031, and a thirdelectronic device1051 within anetwork space1000. Thenetwork space1000 represents, for example, an environment, in which a plurality of electronic devices are linked with each other by using wired or wireless communication to provide information to theuser1010 or obtain information from theuser1010.
• According to an example, first contents1009 (for example, education contents) outputtable through the third electronic device1051 (for example, the monitor951) may be provided through at least a partial region of the object1007 (for example, the monitor object907) disposed within thevirtual environment1005 displayed through a display of the firstelectronic device1001. According to an example, theuser1010 may execute a memo application assecond contents1033 through the secondelectronic device1031. For example, thesecond contents1033 may be provided through theobject1007 together with thefirst contents1009. For example, thefirst contents1009 may be provided through a first region1007-1 of the object1007 (for example, a left portion of the object1007), and thesecond contents1033 may be provided through a second region1007-3 of the object1007 (for example, a right portion of the object1007).
• According to an example, if theuser1010 inputs specific contents through the memo application of the secondelectronic device1031, the specific contents may be displayed through theobject1007. For example, if the secondelectronic device1031 receives a phrase “Hello!” from theuser1010, the firstelectronic device1001 may display the phrase “Hello!”1035 on the second region1007-3 of theobject1007, on which thesecond contents1033 are displayed.
• According to an example, if theuser1010 does not input an additional phrase for a predetermined time or inputs a command that a phrase input is completed, the firstelectronic device1001 may simultaneously display thephrase1035 on theobject1007. For example, the phrase “Hello!”1035 may be simultaneously output through a display of the electronic device1001 (for example, by using one frame provided for one vertical clock). According to various examples, in response to the input of the phrase of theuser1010, the firstelectronic device1001 may instantly display thephrase1035 through the display. For example, if theuser1010 inputs “H” into the secondelectronic device1031, the firstelectronic device1001 may output “H” through the display in response to the input of “H”. Next, if theuser1010 sequentially inputs “e”, “l”, “I”, “o”, and “!”, the firstelectronic device1001 may sequentially display the respective characters through the display in response to the respective inputs.
• According to an example, an actual use environment of an externalelectronic device1031, on which thephrase1035 is displayed, may correspond to thevirtual environment1005 displayed on themonitor object1007. For example, if thephrase1035 is displayed on a center region of the memo application, thephrase1035 may be displayed on a center region of the second region1007-3 of themonitor object1007. If thephrase1035 is input with a first size through the externalelectronic device1031, thephrase1035 may be displayed with a second size (for example, about two times of the first size) corresponding to a ratio of the first size on themonitor object1007. For example, if thephrase1035 is input larger through the externalelectronic device1031, the phrase may be displayed larger on the second region1007-3 of themonitor object1007.
• According to an example, the firstelectronic device1001 may display at least some contents1009-1 of thefirst contents1009 output through the first region1007-1 of theobject1007 on thesecond contents1033 output through the second region1007-3 of theobject1007 automatically or based on a user's input. For example, the firstelectronic device1001 may automatically back up at least some contents1009-1 in thesecond contents1033 based on various cases, that is, a case where at least some contents1009-1 include a specific term designated by theuser1010, importance of at least some contents1009-1 is relatively higher than that of other contents, and at least some contents1009-1 are not properly displayed through theobject1007.
• According to an example, if at least some contents1009-1 are displayed through the second region1007-3 of theobject1007, on which the second contents1003 are displayed, at least some contents1009-1 may also be displayed on a display of the secondelectronic device1031. According to an example, the firstelectronic device1001 may transmit at least some contents1009-1 to thesecond contents1033, so that at least some contents1009-1 may be displayed or stored through the secondelectronic device1031.
• According to an example, if at least some contents1009-1 are displayed through thesecond contents1033, the thirdelectronic device1051 may haveadditional information1053 for at least some contents1009-1. For example, the thirdelectronic device1051 may obtain information indicating that at least some contents1009-1 are displayed or stored through the secondelectronic device1031 as theadditional information1053 for at least some contents1009-1 from the firstelectronic device1001 or the secondelectronic device1031. According to an example, if theuser1010 selects at least some contents1009-1 based on the gaze or a gesture of theuser1010, the thirdelectronic device1051 may display theadditional information1053 through a display of the thirdelectronic device1051.
• As described above, within thenetwork space1000, information provided through at least one of the firstelectronic device1001 to the thirdelectronic device1051 may be shared through another electronic device, and the information obtained through at least one of the firstelectronic device1001 to the thirdelectronic device1051 also be displayed or stored through another electronic device.
• FIG. 11 illustrates an example of a user interface (for example, the virtual environment500) provided through an electronic device (for example, the electronic device101). Referring toFIG. 11, for example, in order to form avirtual environment1105 corresponding to aliving room1130, auser1110 may scan theliving room1130 in a predetermined direction1135 (for example, from the left to the right) while wearing an electronic device1101 (for example, the electronic device101).
• For example, anelectronic flowerpot1141, adesktop computer1143, amonitor1145, and akeyboard1147 disposed in theliving room1130 may be discovered by theelectronic device1101, and displayed as first to fourth objects1141-1,1143-1,1145-1, and1147-1 at locations corresponding to actual spaces thereof, respectively, within thevirtual environment1105.
• According to an example, theelectronic device1101 may differently displayuser interfaces1150 and1170 according to a processing degree of thevirtual environment1105 in generating thevirtual environment1105. Thefirst user interface1150 may, for example, vary a transparency of thevirtual environment1105 according to the processing degree. For example, if the processing degree is about 10%,transparency1151 of thevirtual environment1105 may have a first transparency having the most unclear transparency. If the processing degree is about 50%,transparency1153 of thevirtual environment1105 may have a second transparency having a middle transparency higher than the first transparency. If the processing degree is about 80%,transparency1155 of thevirtual environment1105 may have third transparency having the highest transparency. According to an example, the transparency of thevirtual environment1105 provided by thefirst user interface1150 may be gradually changed from about 0% to about 100% according to a processing degree.
• Thesecond user interface1170 may, for example, vary a region of thevirtual environment1105 output through a display of theelectronic device1101 based on the processing degree. For example, if a processing degree is about 10%, aregion1171 of thevirtual environment1105 shown to theuser1110 may have a first region that is the narrowest region. If the processing degree is about 50%, thetransparency1173 of thevirtual environment1105 may have a second region having a middle region wider than the first region. If the processing degree is about 80%,transparency1175 of thevirtual environment1105 may have a third region having the widest region. According to an example, the region of thevirtual environment1105 provided by thesecond user interface1170 may be gradually changed from about 0% to about 100% based on a processing degree.
• InFIG. 11, as an example, the processing degree of thevirtual environment1105 is represented by transparency or a shown region, but according to various examples, the processing degree of thevirtual environment1105 is variously represented by chroma, luminance, brightness, color, light and shade, shaking, movement, or the like.
• According to various examples, an electronic device (for example, the electronic device101) for sharing information through a virtual environment (for example, the virtual environment500) may include a display (for example, the display170) and an information providing module, e.g., in the form of processing circuitry, (for example, the information providing module110) functionally connected with the display, and the information providing module may display an object corresponding to an external electronic device for the electronic device through the display, obtain information to be output through the external electronic device, and provide contents corresponding to the information in relation to a region, on which the object is displayed.
• According to various examples, the information providing module may check a plurality of external electronic devices for the electronic device, and select the external electronic device among the plurality of external electronic devices based on a user's input or functions of the plurality of external electronic devices.
• According to various examples, the information providing module may check a plurality of external electronic devices for the electronic device, and determine a currently operated electronic device among the plurality of external electronic devices as the external electronic device.
• According to various examples, at least one object may include at least one of an image for an appearance of the external electronic device, an icon corresponding to the external electronic device, and text corresponding to the external electronic device.
• According to various examples, the information providing module may obtain a relative location of the external electronic device with respect to the electronic device, and if the relative location is a first location, the information providing module may display the object on a first region of the display, and if the relative location is a second location, the information providing module may display the object on a second region of the display.
• According to various examples, the information providing module may check a time, for which the gaze of a user for the electronic device stays on the object, and if the time is a first time, the information providing module may display the object in a first format, and if the time is a second time, the information providing module may display the object in a second format.
• According to various examples, the information providing module may provide the contents to at least a partial region of the region.
• According to various examples, the contents include a sound, and the information providing module may check a location of the region in the display, and provides the sound in a first direction with respect to the electronic device if the location is a first location, and the information providing module may provide the sound in a second direction with respect to the electronic device if the location is a second location.
• According to various examples, the information providing module may obtain another information obtained through another external electronic device for the electronic device, and further provide other contents corresponding to another information.
• According to various examples, the information providing module may provide the contents if the electronic device is worn the user.
• According to various examples, the contents may be provided to the user through at least one of the electronic device and the external electronic device.
• According to various examples, the information providing module may control the external electronic device so that the contents are prevented from being provided through the external electronic device any longer if the electronic device is attached onto the user.
• According to various examples, the information providing module may control the external electronic device so that the contents are provided to the user through the external electronic device based on the fact that the electronic device is detached from the user of the electronic device.
• According to various examples, the information providing module may stop providing the contents based on external environment information about the electronic device.
• According to various examples, an electronic device for sharing information through a virtual environment may include a display and an information providing module functionally connected with the electronic device, and the information providing module may display a plurality of objects corresponding to a plurality of external electronic devices, respectively, for the electronic device through the display, select at least one object among the plurality of objects, obtain information to be output by at least one external electronic device corresponding to at least one object among the plurality of external electronic devices, and provide the contents corresponding to the information in relation to a region, on which at least one object is displayed.
• According to various examples, the information providing module may display a virtual environment through the display, and display the plurality of objects within the virtual environment.
• According to various examples, the information providing module may change the virtual environment to another virtual environment based on at least one input for the electronic device.
• According to various examples, at least one object may be displayed on a designated region of the virtual environment.
• According to various examples, the contents may include first sub contents and second sub contents, and at least one object may include a first object for outputting the first sub contents and a second object for outputting the second sub contents.
• According to various examples, the plurality of objects may include a first object and a second object, and the information providing module may output the first object based on a first frequency and the second object based on a second frequency.
• According to various examples, the plurality of objects may include a first object and a second object, and the information providing module may obtain biometric information about a user of the electronic device, and the information providing module may determine the first object as at least one object if the biometric information corresponds to a first frequency, and determine the second object as at least one object if the biometric information corresponds to a second frequency.
• According to various examples, the information providing module may determine at least one object based on at least one of the gaze and a gesture of the user for the electronic device.
• According to various examples, the information providing module may provide a second part of output contents that is a next part of a first part of the contents previously output through at least one external electronic device as the information.
• According to various examples, the information providing module may display a control menu for controlling the information in relation to at least one content through the region.
• According to various examples, the information providing module may obtain an input for another object among the plurality of objects, and transmit at least a part of at least one content or at least a part of additional information about at least one content to another external electronic device so that at least the part of at least one contents or at least the part of the additional information on at least one contents is provided through another external electronic device corresponding to another object based on the input.
• FIG. 12 is aflowchart1200 illustrating an example method of providing information by an electronic device (for example, the electronic device101). According to an example, inoperation1210, an electronic device (for example, the object display module630) may display an object (for example, the television object525) corresponding to an external electronic device (for example, the television425) for the electronic device. According to an example, the electronic device may provide a virtual environment (for example, the virtual environment500). For example, the electronic device may display the object within the virtual environment.
• Inoperation1230, the electronic device (for example, the information obtaining module640) may obtain information (for example, the video465) to be output through the external electronic device. According to an example, the electronic device may obtain the information if the external electronic device is selected as a target electronic device based on a user's input. Inoperation1250, the electronic device (for example, the content providing module650) may provide contents (for example, the video465) corresponding to the information in relation to a region, on which the object is displayed.
• According to various examples, a method of sharing information through a virtual environment by an electronic device (for example, the electronic device101) may include an operation of displaying an object corresponding to an external electronic device for the electronic device through a display (for example, the display170) functionally connected with the electronic device, an operation of obtaining information to be output through the external electronic device, and an operation of providing contents corresponding to the information in relation to a region, on which the object is displayed.
• According to various examples, the operation of displaying the object may include an operation of checking a plurality of external electronic devices for the electronic device, and an operation of selecting the external electronic device among the plurality of external electronic devices based on a user's input or functions of the plurality of external electronic devices.
• According to various examples, the operation of displaying the object may include an operation of checking a plurality of external electronic devices for the electronic device, and determining a currently operated electronic device among the plurality of external electronic devices as the external electronic device.
• According to various examples, at least one object may include at least one of an image for an appearance of the external electronic device, an icon corresponding to the external electronic device, and text corresponding to the external electronic device.
• According to various examples, the operation of displaying the object may include obtaining a relative location of the external electronic device with respect to the electronic device, an operation of displaying the object on a first region of the display if the relative location is a first location, and an operation of displaying the object on a second region of the display if the relative location is a second location.
• According to various examples, the operation of displaying the object may include an operation of checking a time, for which the gaze of a user for the electronic device stays on the object, an operation of displaying the object in a first format if the time is a first time, and an operation of displaying the object in a second format if the time is a second time.
• According to various examples, the operation of providing the contents may include an operation of providing the contents to at least a partial region of the region.
• According to various examples, the contents include a sound, and the providing of the contents may include an operation of checking a location of the region in the display, an operation of providing the sound in a first direction with respect to the electronic device if the location is a first location, and an operation of providing the sound in a second direction with respect to the electronic device if the location is a second location.
• According to various examples, the method of providing information through the virtual environment may further include an operation of obtaining another information obtained through another external electronic device for the electronic device, and an operation of providing other contents corresponding another information.
• According to various examples, the providing of the contents may include an operation of providing the contents if the electronic device is worn by the user.
• According to various examples, the contents may be provided to the user through at least one of the electronic device and the external electronic device.
• According to various examples, the method of providing information through the virtual environment may further include an operation of controlling the external electronic device so that the contents are prevented from being provided through the external electronic device any longer if the electronic device is attached onto the user.
• According to various examples, the operation of controlling the external electronic device may include an operation of controlling the external electronic device so that the contents are provided to the user through the external electronic device if the electronic device is detached from the user of the electronic device.
• According to various examples, the method of providing information through the virtual environment may further include an operation of stopping providing the contents based on external environment information about the electronic device.
• According to various examples, a method of providing information through a virtual environment may include an operation of displaying a plurality of objects corresponding to a plurality of external electronic devices, respectively, for the electronic device through a display functionally connected with the electronic device, an operation of selecting at least one object among the plurality of objects, an operation of obtaining information to be output by at least one external electronic device corresponding to at least one object among the plurality of external electronic device, and an operation of providing the contents corresponding to the information in relation to a region, on which at least one object is displayed.
• According to various examples, the operation of displaying the plurality of objects may include an operation of displaying a virtual environment through the display, and an operation of displaying the plurality of objects within the virtual environment.
• According to various examples, the method of providing information through the virtual environment may further include an operation of changing the virtual environment to another virtual environment based on at least one input for the electronic device.
• According to various examples, at least one object may be displayed on a designated region of the virtual environment.
• According to various examples, the contents may include first sub contents and second sub contents, and at least one object may include a first object for outputting the first sub contents and a second object for outputting the second sub contents.
• According to various examples, the plurality of objects may include a first object and a second object, and the displaying of the plurality of objects may include an operation of outputting the first object based on a first frequency and an operation of outputting the second object based on a second frequency.
• According to various examples, the plurality of objects may include a first object and a second object, and the operation of obtaining the information may include an operation of obtaining biometric information about a user of the electronic device, and an operation of determining the first object as at least one object if the biometric information corresponds to a first frequency, and an operation of determining the second object as at least one object if the biometric information corresponds to a second frequency.
• According to various examples, the operation of obtaining the information may include an operation of determining at least one object based on at least one of the gaze and a gesture of the user for the electronic device.
• According to various examples, the operation of providing the contents may include an operation of providing a second part of output contents that is a next part of a first part of the contents previously output through at least one external electronic device as the information.
• According to various examples, the operation of providing the contents may include an operation of displaying a control menu for controlling the information in relation to at least one contents through the region.
• According to various examples, the method of providing information through the virtual environment may further include an operation of obtaining an input for another object among the plurality of objects, and an operation of transmitting at least a part of at least one content or at least a part of additional information about at least one content to another external electronic device so that at least the part of at least one contents or at least the part of the additional information on at least one contents is provided through another external electronic device corresponding to another object based on the input.
• According to various examples, in a storage medium storing commands, the commands may be set to make one or more processors perform one or more operations if the commands are executed by the one or more processors, and the one or more operations may include an operation of displaying, by an electronic device (for example, the electronic device101) through a display (for example, the display170) functionally connected with the electronic device, an operation of obtaining information to be output through at least one external electronic device, and an operation of providing contents corresponding to the information, respectively, through a region, on which at least one object is displayed.
• The examples of the disclosure disclosed herein and drawings suggest the examples for plainly explaining the contents of the technology of the disclosure and helping the understanding of the disclosure, and do not limit the scope of the disclosure. Therefore, in addition to the examples disclosed herein, the scope of the various embodiments of the disclosure should be construed to include all modifications or modified forms drawn based on the technical idea of the various embodiments of the disclosure.

Claims (26)

What is claimed is:

1. An electronic device, comprising:

a display; and

an information providing processing circuitry functionally connected with the display,

wherein the information providing processing circuitry is configured to display an object corresponding to an external electronic device for the electronic device through the display, to obtain information to be output through the external electronic device, and to provide contents corresponding to the information in relation to a region, on which the object is displayed.

2. The electronic device ofclaim 1, wherein the information providing processing circuitry is configured to check a plurality of external electronic devices for the electronic device, and to select an external electronic device among the plurality of external electronic devices based on at least one of a received input or functions of the plurality of external electronic devices.

3. The electronic device ofclaim 1, wherein the information providing processing circuitry is configured to check a plurality of external electronic devices for the electronic device, and to determine a currently operated external electronic device among the plurality of external electronic devices as the external electronic device.

4. The electronic device ofclaim 1, wherein at least one object includes at least one of an image corresponding to an appearance of the external electronic device, an icon corresponding to the external electronic device, and text corresponding to the external electronic device.

5. The electronic device ofclaim 1, wherein the information providing processing circuitry is configured to obtain a relative location of the external electronic device with respect to the electronic device, the information providing processing circuitry is configured to display the object on a first region of the display if the relative location is a first location, and the information providing processing circuitry is configured to display the object on a second region of the display if the relative location is a second location.

6. The electronic device ofclaim 1, wherein the information providing processing circuitry is configured to check a time, for which gaze of a user for the electronic device stays on the object, and the information providing processing circuitry is further configured to display the object in a first format if the time is a first time, and the information providing processing circuitry is configured to display the object in a second format if the time is a second time.

7. The electronic device ofclaim 1, wherein the information providing processing circuitry is configured to provide the contents to provide to at least a partial region of the region.

8. The electronic device ofclaim 1, wherein the contents include a sound, and

the information providing processing circuitry is configured to check a location of the region in the display, and to provide the sound in a first direction with respect to the electronic device if the location is a first location, and to provide the sound in a second direction with respect to the electronic device if the location is a second location.

9. The electronic device ofclaim 1, wherein the information providing processing circuitry is configured to obtain another information obtained through another external electronic device for the electronic device, and to provide other contents corresponding to said another information.

10. The electronic device ofclaim 1, wherein the information providing processing circuitry is configured to provide the contents if the electronic device is worn.

11. The electronic device ofclaim 1, wherein the contents are set to be provided through at least one of the electronic device and the external electronic device.

12. The electronic device ofclaim 11, wherein the information providing processing circuitry is configured to control the external electronic device so that the contents are prevented from being provided through the external electronic device if the electronic device is attached onto the user.

13. The electronic device ofclaim 11, wherein the information providing processing circuitry is configured to control the external electronic device so that the contents are provided through the external electronic device if the electronic device is detached from the user of the electronic device.

14. The electronic device ofclaim 1, wherein the information providing processing circuitry is configured to stop providing the contents based on external environment information about the electronic device.

15. A method of sharing information by an electronic device, comprising:

displaying a plurality of objects corresponding to a plurality of external electronic devices for the electronic device, respectively, through a display functionally connected with the electronic device;

selecting one or more objects among the plurality of objects;

obtaining information to be output by one or more external electronic devices corresponding to the one or more objects among the plurality of external electronic devices; and

providing contents corresponding to the information in relation to a region, on which the one or more objects are displayed.

16. The method ofclaim 15, wherein displaying comprises:

displaying a virtual environment through the display; and

displaying the plurality of objects within the virtual environment.

17. The method ofclaim 16, further comprising:

changing the virtual environment to another virtual environment based on at least one input for the electronic device.

18. The method ofclaim 16, wherein the one or more objects are displayed on a designated region of the virtual environment.

19. The method ofclaim 15, wherein the contents include first sub contents and second sub contents, and the one or more objects include a first object for outputting the first sub contents and a second object for outputting the second sub contents.

20. The method ofclaim 15, wherein the plurality of objects includes a first object and a second object, and

displaying comprises:

outputting the first object based on a first frequency; and

outputting the second object based on a second frequency.

21. The method ofclaim 15, wherein the plurality of objects includes a first object and a second object, and

obtaining comprises:

obtaining biometric information;

determining the first object as the one or more objects if the biometric information corresponds to the first frequency; and

determining the second object as the one or more objects if the biometric information corresponds to the second frequency.

22. The method ofclaim 15, wherein obtaining comprises: determining the one or more objects based on at least one of a gaze and a gesture of the user of the electronic device.

23. The method ofclaim 15, wherein providing comprises: providing a second part of output contents that is a next part of a first part of the contents previously output through the one or more external electronic devices as the information.

24. The method ofclaim 15, wherein providing comprises: displaying a control menu for controlling the information in relation to the one or more contents through the region.

25. The method ofclaim 15, further comprising:

obtaining an input for another object among the plurality of objects; and

transmitting at least a part of the one or more contents or additional information about the one or more contents to another external electronic device so that the part of the one or more contents or the additional information about the one or more contents are provided to another external electronic device corresponding to another object.

26. A computer readable recording medium recording a program for executing: displaying one or more objects corresponding to one or more external electronic devices for the electronic device through a display functionally connected with the electronic device;

obtaining information to be output through the one or more external electronic devices; and

providing contents corresponding to the information, respectively, in relation to a region, on which the one or more objects are is displayed.

Images