US20130093719A1 - Information processing apparatus - Google Patents

Abstract

An information processing apparatus comprising: a touch panel that detects a contact operation and a contactless operation of an operation element by detecting an electrostatic capacitance of an operation surface of the touch panel; and a controller that determines whether an application activated at the information processing apparatus supports contactless operations; controls the application based only on contact operations detected at the touch panel when is determined that the application does not support contactless operations; and controls the application based on both contact operations and contactless operations detected at the touch panel when it is determined that the application supports contactless operations.

Description

CROSS REFERENCE TO RELATED APPLICATION
• The present application claims the benefit of the earlier filing date of U.S. Provisional Patent Application Ser. No. 61/547,783 filed on Oct. 17, 2011, the entire contents of which is incorporated herein by reference.
BACKGROUND
• 1. Field of the Disclosure
• The present disclosure relates to an information processing apparatus applied to an electronic device provided with an operable unit that detects an operation mode on the basis of changes in the capacitance of an operable surface, such as a touch panel, for example.
• Specifically, the present disclosure relates to an information processing apparatus applied to an electronic device such as a mobile phone, a PHS phone (PHS: Personal Handyphone System), a PDA (PDA: Personal Digital Assistant), a digital camera, a digital video camera, a portable game console, or a notebook computer, for example.
• 2. Description of Related Art
• At present, mobile phones provided with capacitive touch panels are known. In such mobile phones, a controller detects a direct touch operation via the touch panel on the basis of an input processing program. Then, the controller controls execution of a process corresponding to the detected direct touch operation from among various information processes in a given application program.
• FIG. 10 illustrates a function block diagram of such a controller realized as a result of a controller acting on the basis of the input processing program.
• As illustrated inFIG. 10, in the case of acting on the basis of the input processing program, the controller functions as a two-dimensional coordinate converter101 which detects a direct touch operation position on atouch panel100 and outputs two-dimensional coordinate information (X coordinate information and Y coordinate information) corresponding to the direct touch operation position.
• Herein, in the case of functioning as the two-dimensional coordinate converter101, the controller may also create two-dimensional coordinate information corresponding to a direct touch operation position on the basis of output from thetouch panel100, which is hardware (HW) as discussed above. Alternatively, before functioning as the two-dimensional coordinate converter101, the controller may function as firmware (FW) which detects two-dimensional coordinate information corresponding to a direct touch operation position on the basis of output from the hardware (HW)touch panel100, and subsequently function as the two-dimensional coordinate converter101 and create two-dimensional coordinate information corresponding to the direct touch operation position.
• Also, in the case of acting on the basis of the input processing program, the controller functions as anoperation detector104 in awindow manager102 which detects a direct touch operation position and a direct touch operation state (direct touch operation mode) on the basis of the two-dimensional coordinate information created when the controller functions as the two-dimensional coordinate converter101.
• Then, the controller functions as anapplication execution controller103 that controls execution of information processing which, from among the various information processing of a currently activated application program, corresponds to a direct touch operation state, etc. detected when the controller functions as theoperation detector104 of thewindow manager102.
• FIG. 11 illustrates a flowchart of a controller's input processing action based on the input processing program. Thetouch panel100 is configured to identify an operation position by detecting changes in the capacitance between a user's finger and theconductive film151 of the two-dimensional coordinate converter101, as illustrated inFIG. 12A.
• In step S100 of the flowchart inFIG. 11, the controller acquires (scans) the capacitance values of all sensors on thetouch panel100 by functioning as the two-dimensional coordinate converter101. Also, in step S100 the controller determines whether or not capacitance values have been detected for all sensors on thetouch panel100, and advances the process to step S101 when capacitance values have been detected for all sensors.
• If the user performs a direct touch operation with his or her finger on theconductive film151 forming the operable surface of thetouch panel100, a change in capacitance values occurs centered about the direct touch operation position on theconductive film151, as illustrated inFIG. 12B. In other words, in the case where a direct touch operation is conducted on thetouch panel100, the capacitance values change such that the approximate center of the direct touch operation position on theconductive film151 becomes the peak value of the capacitance values, with the capacitance values gradually becoming smaller away from the center of the direct touch operation position.
• In step S101 the controller, by functioning as the two-dimensional coordinate converter101, compares the capacitance values of the individual sensors to a direct touch threshold level as illustrated inFIG. 12B, and determines whether or not there exist sensors among all sensors on thetouch panel100 having capacitance values equal to or greater than the direct touch threshold level.
• In the case where it is determined that sensors having capacitance values equal to or greater than the direct touch threshold level do not exist, the controller advances the process to step S104 and determines whether or not to continue detecting the capacitance values of the respective sensors. Then, in the case where it is determined not to continue detecting the capacitance values of the respective sensors, the controller ends the process of the flowchart inFIG. 11. Meanwhile, in the case where it is determined to continue detecting the capacitance values of the respective sensors, the controller returns the process to step S100 at the timing for conducting the next detection of the capacitance values of the respective sensors and acquires the capacitances value of the respective sensors again.
• In contrast, in the case where it is determined in step S101 that there do exist sensors having capacitance values equal to or greater than the direct touch threshold level, the controller advances the process to step S102, identifies the sensors having capacitance values equal to or greater than the direct touch threshold level, and advances the process to step S103.
• Then, in step S103 the controller creates two-dimensional coordinate information (i.e., respective X-axis and Y-axis coordinate information) corresponding to the placement positions on thetouch panel100 of the identified sensors having capacitance values equal to or greater than the direct touch threshold level, controls storage of the two-dimensional coordinate information to a register, and advances the process to step S104.
• The controller is configured to control storage of two-dimensional coordinate information thus created to a register every time the capacitance values of respective sensors are acquired from thetouch panel100 and the two-dimensional coordinate information is created. For this reason, respective sets of two-dimensional coordinate information in accordance with direct touch operations on thetouch panel100 by the user are successively stored in the register. Consequently, direct touch operation history information (i.e., the respective sets of two-dimensional coordinate information) corresponding to direct touch operations on thetouch panel100 by the user is stored in the register.
• When functioning as theoperation detector104 of thewindow manager102, the controller detects the user's direct touch operation mode, like that illustrated inFIG. 12C for example, on the basis of the direct touch history given by the respective sets of two-dimensional coordinate information stored in the register.
• Then, when functioning as theapplication execution controller103, the controller controls execution of information processing which, from among the various information processing of a currently activated application program, corresponds to the detected direct touch operation mode and the two-dimensional coordinate information used when detecting the direct touch operation mode.
• By determining to continue detecting capacitance values in step S104 and returning the process to step S100, the controller acquires the capacitance values of the respective sensors in thetouch panel100 at given time intervals, and conducts processing for creating the two-dimensional coordinate information, detecting the direct touch operation mode, and executing an application program based on the detected operation mode, etc., on the basis of the acquired capacitance values.
SUMMARY
• By performing a direct touch operation on thetouch panel100 with an operating object such as a finger as discussed above, information processing operations based on the currently activated application program become possible.
• However, the present inventor has recognized the need for an information processing apparatus in which information processing in a given application program is operable not only by direct touch operations, but also by contactless operations.
• According to one exemplary embodiment, the disclosure is directed to an information processing apparatus comprising: a touch panel that detects a contact operation and a contactless operation of an operation element by detecting an electrostatic capacitance of an operation surface of the touch panel; and a controller that determines whether an application activated at the information processing apparatus supports contactless operations; controls the application based only on contact operations detected at the touch panel when is determined that the application does not support contactless operations; and controls the application based on both contact operations and contactless operations detected at the touch panel when it is determined that the application supports contactless operations.
• According to another exemplary embodiment, the disclosure is directed to an information processing method performed by an information processing apparatus including a touch panel that detects a contact operation and a contactless operation of an operation element by detecting an electrostatic capacitance of an operation surface of the touch panel, the method comprising: determining whether an application activated at the information processing apparatus supports contactless operations; controlling the application based only on contact operations detected at the touch panel when is determined that the application does not support contactless operations; and controlling the application based on both contact operations and contactless operations detected at the touch panel when it is determined that the application supports contactless operations.
• According to another exemplary embodiment, the disclosure is directed to a non-transitory computer-readable medium including computer program instructions, which when executed by an information processing apparatus including a touch panel that detects a contact operation and a contactless operation of an operation element by detecting an electrostatic capacitance of an operation surface of the touch panel, cause the information processing apparatus to perform a method, the method comprising: determining whether an application activated at the information processing apparatus supports contactless operations; controlling the application based only on contact operations detected at the touch panel when is determined that the application does not support contactless operations; and controlling the application based on both contact operations and contactless operations detected at the touch panel when it is determined that the application supports contactless operations.
• According to an embodiment of the present disclosure, new operation modes can be provided in which information processing in a given application program is operable by contactless operations.
BRIEF DESCRIPTION OF DRAWINGS
• FIG. 1 is a block diagram of a mobile phone according to an embodiment implementing the present disclosure.
• FIG. 2 is a function block diagram during input processing by a controller in a mobile phone according to an embodiment.
• FIG. 3 is a flowchart illustrating input processing action in a mobile phone according to an embodiment.
• FIGS. 4A to 4C are diagrams for explaining a direct touch threshold level and a contactless threshold level provided in a mobile phone according to an embodiment.
• FIG. 5 is a diagram illustrating exemplary capacitance values of respective sensors during a direct touch operation in a mobile phone according to an embodiment.
• FIG. 6 is a diagram illustrating exemplary capacitance values of respective sensors during an contactless operation in a mobile phone according to an embodiment.
• FIG. 7 is a flowchart for explaining action for creating two-dimensional coordinate information for a contactless operation in a mobile phone according to an embodiment.
• FIGS. 8A to 8C are diagrams illustrating an exemplary transition pattern across capacitance maps during a contactless operation in a mobile phone according to an embodiment.
• FIG. 9 is a timing chart for explaining information processing of a telephony application conducted by a direct touch operation and a contactless operation in a mobile phone according to an embodiment.
• FIG. 10 is a function block diagram during input processing by a controller in a conventional mobile phone.
• FIG. 11 is a flowchart illustrating input processing action in a conventional mobile phone.
• FIGS. 12A to 12C are diagrams for explaining a direct touch threshold level provided in a conventional mobile phone.
DETAILED DESCRIPTION
• The present disclosure can be applied to a mobile phone as one example.
[Configuration of Mobile Phone]
• FIG. 1 is a block diagram of a mobile phone according to an embodiment of the present disclosure. As illustrated inFIG. 1, a mobile phone according to an embodiment includes anantenna1 and acommunication circuit2 which conduct audio telephony, video telephony, and wireless communication such as email and Web data (Web: World Wide Web) with a base station.
• Also, the mobile phone includes afirst speaker unit3a(internal speaker unit) for obtaining acoustic output such as telephone receiver audio at a volume hearable when the mobile phone is brought close to the ear, as well as asecond speaker unit3b(external speaker unit) for obtaining acoustic output such as telephone receiver audio in cases where the mobile phone is used at a position distanced from the user's ear, such as during hands-free telephony or during video telephony.
• The mobile phone also includes amicrophone unit4 for picking up telephone transmitter audio, etc., adisplay unit5 forming what is called a touch panel for performing direct touch operations and contactless operations thereon, and a plurality ofhardware keys6 physically provided on the chassis of the mobile phone.
• The mobile phone also includes a light emitter7 (LED: Light Emitting Diode) for notifying the user of incoming/outgoing signals, etc. with light, afirst camera unit8afor shooting a still image or video of a desired subject, and asecond camera unit8bfor shooting the user, etc. of the mobile phone during video telephony, for example.
• The mobile phone also includes avibration unit9 for notifying the user of incoming/outgoing signals, etc. by causing the chassis of the mobile phone to vibrate, and atimer10 that keeps the current time.
• The mobile phone also includes anacceleration sensor11 for detecting shake operations, etc. imparted to the chassis of the mobile phone, as well as a GPS antenna12 (GPS: Global Positioning System) and aGPS unit13 for detecting the present location of the mobile phone and the shooting location of still images or videos shot primarily with thefirst camera unit8a.
• The mobile phone also includesmemory14 storing a communication program for conducting the wireless communication processing via a base station and various application programs in addition to various data handled by these various application programs, and acontroller15 that controls overall action of the mobile phone.
• The mobile phone also includes an operation history register16 that stores a capacitance map which indicates the capacitance values of all sensors in thedisplay unit5 forming the touch panel.
• Theoperation history register16 includes a storage area for direct touch operations and a storage area for contactless operations. It is configured such that direct touch operation positions and direct touch operation-induced capacitance values corresponding to direct touch operations by the user are successively stored in the storage area for direct touch operations in theoperation history register16. It is also configured such that capacitance values corresponding to contactless operations by the user are successively stored in the storage area for contactless operations in theoperation history register16.
• The mobile phone also includes contactless operationmode file memory17 storing contactless operation mode files for detecting contactless operation modes (contactless operation patterns) for respective contactless operations.
• In the case of the mobile phone in this embodiment, an application program that allows contactless operations includes a contactless operation mode file in which contactless operation mode data corresponding to various processes in the application program is stored as a file.
• In other words, the contactless operation mode data is data expressing transition patterns on the capacitance map for respective contactless operation modes corresponding to respective processes in the application program. Furthermore, in the contactless operation mode file, contactless operation mode data for the respective contactless operation modes is formed as a file.
• Stated differently, in the case of the mobile phone in this embodiment, an application program that allows contactless operations includes a contactless operation mode file, in which the capacitance map transition patterns for each of the contactless operations corresponding to respective processes in that application program are stored in a file as the contactless operation mode data.
• Thecontroller15 reads out a contactless operation mode file attached to the application program at given timings, such as when activating the application program, for example, and loads it into the contactless operationmode file memory17.
• Thecontroller15 compares a capacitance map transition pattern detected during a contactless operation to the capacitance map transition patterns expressed by the respective contactless operation mode data in the contactless operation mode file for that application program which was loaded into the contactless operationmode file memory17, and detects the contactless operation mode. It is configured such that thecontroller15 then controls execution of a process corresponding to the detected contactless operation mode on the basis of the activated application program.
• Next, the mobile phone in this embodiment includesthreshold level memory18 in which the direct touch threshold level and the contactless threshold level are respectively stored.
• Thecontroller15 is provided with a first control mode that controls execution of processes in an application program according to direct touch operations, and a second control mode that controls execution of processes in an application program according to direct touch operations and contactless operations.
• Additionally, in the case where it is determined that the activated application program does not support contactless operations, thecontroller15 switches to the first control mode and controls execution of processes in the activated application program according to direct touch operations detected on the basis of the direct touch threshold level stored in thethreshold level memory18.
• Meanwhile, in the case where it is determined that the activated application program does support contactless operations, thecontroller15 switches to the second control mode and controls execution of processes in the activated application program according to direct touch operations detected on the basis of the direct touch threshold level stored in thethreshold level memory18, while also controlling execution of processes in the activated application program according to contactless operations detected on the basis of the contactless threshold level stored in thethreshold level memory18.
• Next, in addition to the communication program, an input processing program for conducting information processing of direct touch operations and contactless operations on thedisplay unit5 forming the touch panel (input processing) is stored in thememory14.
• Also stored in thememory14 is a camera control program for controlling the shooting of still images or video with therespective camera units8aand8b. The camera control program is provided with a viewer program for displaying shot still images on thedisplay unit5, etc. The viewer program is provided with functions for changing the display magnification by enlarging or reducing the displayed image, and facial recognition functions for detecting facial images of subjects (persons) appearing in still images.
• The camera control program is also provided with a video playback program for displaying shot videos on thedisplay unit5, etc. Also, the video playback program is provided with playback speed modification functions for controlling changes to the video playback speed.
• Also stored in thememory14 are an email management program for controlling the creation and transmitting/receiving of email, and a scheduler management program for managing a scheduler in which the user's schedule is registered.
• Also stored in thememory14 are a Web browsing program for viewing Web pages by transmitting/receiving information by accessing a server provided on a given network such as a communication network or the Internet, a contacts list management program for managing a contacts list that registers personal information such as the names, addresses, telephone numbers, email addresses, and facial photos of friends and acquaintances (i.e., the contacts list is a personal information registration area), and a music player program for playing back music data.
• Also stored in thememory14 is a lock/unlock application program for realizing an unlocked mode that receives direct touch operations and contactless operations on thedisplay unit5 and executes actions according to the operations, as well as a locked mode that cancels direct touch operations and contactless operations on thedisplay unit5.
• Also stored in thememory14 is a scheduler in which the user's desired schedule is registered (i.e., a schedule data registration area), and a contacts list in which information such as the user names, still images (facial images, etc.), addresses, telephone numbers, email addresses, and birthdates of the user's friends and acquaintances is registered (i.e., a personal information registration area for respective users).
• Also stored in thememory14 are music data played back by the music player program, still image data and video data played back by the viewer program and video playback program in the camera control program, transmitted/received email data, and a history of transmitted/received telephone calls and emails.
• A projected capacitive touch panel is provided as the touch panel of thedisplay unit5.
• The projected capacitive touch panel includes a resistive film with an electrode layer underneath, as well as a substrate layer provided with a control IC (control integrated circuit). In the electrode layer underneath the resistive film, many electrode patterns forming a mosaic consisting of two layers (horizontal and vertical) are arranged on a glass, plastic, or other substrate using transparent electrodes such as indium tin oxide (ITO).
• The projected capacitive touch panel identifies an operation position by detecting changes in electrode capacitance due to a direct touch operation or a contactless operation from two (horizontal and vertical) electrode lines. By respectively providing many electrode lines in the horizontal and vertical directions, multipoint detection of direct touch operations becomes possible.
• Although a projected capacitive touch panel is provided as thedisplay unit5 in this example, what is called a surface capacitive touch panel may also be provided instead of the projected capacitive touch panel.
[Function Block Configuration]
• By acting on the basis of an input processing program stored in thememory14, thecontroller15 of the mobile phone detects a “direct touch operation” made by causing a finger or other operating element (conductive member) to contact the operable surface of thedisplay unit5 which forms a projected capacitive touch panel.
• The mobile phone is also configured to detect a “contactless operation” in which a finger or other conductive member is moved over the operable surface of thedisplay unit5 such that the distance between the operable surface of thedisplay unit5 and the finger is short enough to cause at least a given change in the capacitance of thedisplay unit5, but without the finger directly touching the operable surface.
• Thecontroller15 is also provided with a “first control mode” that controls execution of processes in an application program according to direct touch operations, and a “second control mode” that controls execution of processes in an application program according to both direct touch operations and contactless operations.
• Additionally, in the case where it is determined that the activated application program does not support contactless operations, thecontroller15 switches to the first control mode and controls execution of processes in the activated application program according to direct touch operations detected on the basis of the direct touch threshold level stored in thethreshold level memory18.
• Meanwhile, in the case where it is determined that the activated application program does support contactless operations, thecontroller15 switches to the second control mode and controls execution of processes in the activated application program according to direct touch operations detected on the basis of the direct touch threshold level stored in thethreshold level memory18, while also controlling execution of processes in the activated application program according to contactless operations detected on the basis of the contactless threshold level stored in thethreshold level memory18.
• FIG. 2 illustrates function blocks of thecontroller15 which are realized by thecontroller15 acting on the basis of the input processing program.
• When acting on the basis of the input processing program, thecontroller15 switches to the first control mode in the case where a contactless operation mode file discussed earlier is not attached to the activated application program.
• Upon switching to the first control mode, thecontroller15 functions as theread controller19 illustrated inFIG. 2, reads out the capacitance values of all capacitive sensors in thedisplay unit5, and compares the respective capacitance values to the direct touch threshold level stored in thethreshold level memory18. In so doing, thecontroller15 creates two-dimensional coordinate information for thedisplay unit5 corresponding to the position and number of direct touches by the user (i.e., the number of fingers in a direct touch operation).
• Also, upon switching to the first control mode, thecontroller15 functions as atouch driver21, and stores two-dimensional coordinate information corresponding to the detected direct touch operation in the storage area for direct touch operations in theoperation history register16.
• Also, upon switching to the first control mode, thecontroller15 functions as anoperation detector26 of awindow manager23, and detects a direct touch operation mode (direct touch operation pattern) and a direct touch operation position on the basis of the two-dimensional coordinate information corresponding to the direct touch operation acquired when thecontroller15 functioned as thetouch driver21, as well as two-dimensional coordinate information obtained during past direct touch operations being stored in the storage area for direct touch operations in theoperation history register16.
• Additionally, in the first control mode, thecontroller15 functions as anapplication execution controller24, and controls execution of a process from among respective processes in the currently active application program that corresponds to the detected direct touch operation mode (direct touch operation pattern) and direct touch operation position.
• In contrast, when acting on the basis of the input processing program, thecontroller15 switches to the second control mode in the case where a contactless operation mode file discussed earlier is attached to the activated application program.
• Upon switching to the second control mode, thecontroller15 functions as acontactless operation enabler25 of thewindow manager23, and reads the contactless operation mode file attached to the activated application program. Then, thecontroller15 functions as acontactless operation detector20, and loads the contactless operation mode file read above into the contactless operationmode file memory17.
• Also, upon switching to the second control mode, thecontroller15 functions as theread controller19, reads out the capacitance values of all capacitive sensors in thedisplay unit5, and by comparing the respective capacitance values to the direct touch threshold level and the contactless threshold level stored in thethreshold level memory18, determines whether a direct touch operation or a contactless operation has been performed by the user. Then, in the case where it is determined that a direct touch operation has been performed by the user, thecontroller15 functions as theread controller19 to create two-dimensional coordinate information for thedisplay unit5 corresponding to the position and number of direct touches by the user (i.e., the number of fingers in a direct touch operation) as discussed earlier, on the basis of the capacitance values for all capacitive sensors that were read out from thedisplay unit5 as well as the direct touch threshold level stored in thethreshold level memory18.
• Also, in the second control mode, thecontroller15 functions as thetouch driver21, and stores two-dimensional coordinate information corresponding to the detected direct touch operation in the storage area for direct touch operations in theoperation history register16.
• Also, in the second control mode, thecontroller15 functions as theoperation detector26 of thewindow manager23, and detects a direct touch operation mode (direct touch operation pattern) and a direct touch operation position on the basis of the two-dimensional coordinate information corresponding to the direct touch operation acquired when thecontroller15 functioned as thetouch driver21, as well as two-dimensional coordinate information obtained during past direct touch operations being stored in the storage area for direct touch operations in theoperation history register16.
• Additionally, in the second control mode, thecontroller15 functions as theapplication execution controller24, and controls execution of a process from among the respective processes in the currently active application program that corresponds to the detected direct touch operation mode (direct touch operation pattern) and direct touch operation position.
• Also, in the second control mode, in the case where thecontroller15 functions as theread controller19 and thereby determines that a contactless operation has been performed by the user, thecontroller15 functions as thecontactless operation detector20, creates a capacitance map indicating the capacitance values of all capacitive sensors that have been read out from thedisplay unit5, and controls storage of it in the storage area for contactless operations in theoperation history register16.
• Also, in the second control mode, thecontroller15 functions as thecontactless operation detector20, and detects the contactless operation mode (contactless operation pattern) of the contactless operation performed by the user on the basis of a plurality of capacitance maps stored in theoperation history register16. Then, thecontroller15 functions as thecontactless operation detector20, and on the basis of the contactless operation file stored in the contactless operationmode file memory17, creates two-dimensional coordinate information corresponding to the process from among respective processes in the activated application program for the contactless operation pattern that has been detected on the basis of the plurality of capacitance maps.
• Also, in the second control mode, thecontroller15 functions as theoperation detector26 of thewindow manager23, and identifies the contactless operation mode (contactless operation pattern) and contactless operation position on the basis of the created two-dimensional coordinate information.
• Then, in the second control mode, thecontroller15 functions as theapplication execution controller24, and controls execution of a process from among respective processes in the currently active application program that corresponds to the identified contactless operation mode (contactless operation pattern) and contactless operation position.
[Input Processing Action]
• FIG. 3 illustrates a flowchart of input processing action by thecontroller15 based on the input processing program. When an application program is activated, thecontroller15 starts the process illustrated by the flowchart inFIG. 3 on the basis of an input processing program stored in thememory14.
• Among the application programs executed in the mobile phone in this embodiment, application programs able to support contactless operations keep information indicating contactless operation modes (gestures) corresponding to respective information processing as a file. For this reason, when such an application program is activated, in step S1 thecontroller15 functions as theapplication execution controller24 and determines whether or not the activated application program has a contactless operation mode file, thereby determining whether or not the activated application program is able to support contactless operations.
• In the case where a contactless operation mode file is not attached to the activated application program, thecontroller15 switches to the first control mode and executes the processing from step S11 to step S16.
• In contrast, in the case where a contactless operation mode file is attached to the activated application program, thecontroller15 switches to the second control mode and executes the processing from step S2 to step S10 as well as the processing from step S31 to step S34 in the flowchart inFIG. 7.
[Action During First Control Mode]
• First, if thecontroller15 switches to the first control mode because a contactless operation mode file is not attached to the activated application program, thecontroller15 advances the process to step S11, acquires the capacitance values of all capacitive sensors in thedisplay unit5 by functioning as theread controller19, and advances the process to step S12.
• In step S12, thecontroller15 compares the acquired capacitance values to a direct touch threshold level indicated by the broken line inFIG. 4A which is stored in thethreshold level memory18. In so doing, thecontroller15 determines whether or not there exist capacitive sensors that detected capacitance values equal to or greater than the direct touch threshold level. Then, thecontroller15 advances the process to step S13 in the case where it is determined that there do exist capacitive sensors that detected capacitance values equal to or greater than the direct touch threshold level, and advances the process to step S16 in the case where it is determined that there do not exist capacitive sensors that detected capacitance values equal to or greater than the direct touch threshold level.
• In the case of the mobile phone in this embodiment, the capacitance values of all capacitive sensors in thedisplay unit5 are acquired at given time intervals while an application program is being executed. For this reason, if thecontroller15 advances the process to step S16 due to determining that there do not exist capacitive sensors that detected capacitance values equal to or greater than the direct touch threshold level, thecontroller15 functions as theread controller19 and determines whether or not an application program is currently running. In so doing, thecontroller15 determines whether or not to continue detecting capacitance values.
• Then, in cases where it is determined that acquisition of capacitance values has been terminated, such as when an operation for ending the currently active application is performed, for example, thecontroller15 ends all processing in the flowchart illustrated inFIG. 3.
• In contrast, if thecontroller15 advances the process to step S13 due to determining that there do exist capacitive sensors that detected capacitance values equal to or greater than the direct touch threshold level, thecontroller15 functions as theread controller19 and compares the acquire capacitance values to the direct touch threshold level stored in thethreshold level memory18. In so doing, thecontroller15 creates two-dimensional coordinate information for thedisplay unit5 corresponding to the position and number of direct touches by the user (i.e., the number of fingers in a direct touch operation), and advances the process to step S14.
• If two-dimensional coordinate information corresponding to a direct touch operation is created, thecontroller15 functions as thetouch driver21 and stores the two-dimensional coordinate information corresponding to a direct touch operation in the storage area for direct touch operations in theoperation history register16.
• Next, upon advancing the process to step S14, thecontroller15 functions as theoperation detector26 of thewindow manager23 and detects a direct touch operation mode (direct touch operation pattern) and a direct touch operation position on the basis of the two-dimensional coordinate information corresponding to the direct touch operation acquired when thecontroller15 functioned as thetouch driver21, as well as two-dimensional coordinate information obtained during past direct touch operations being stored in the storage area for direct touch operations in theoperation history register16. Thecontroller15 then advances the process to step S15.
• In step S15, thecontroller15 functions as theapplication execution controller24, controls execution of a process from among respective processes in the currently active application program that corresponds to the detected direct touch operation mode (direct touch operation pattern) and direct touch operation position, and advances the process to step S16.
• In step S16, thecontroller15 functions as theread controller19 as discussed above and determines whether or not an application program is currently running. In so doing, thecontroller15 determines whether or not to continue detecting capacitance values.
• Then, in cases where it is determined that acquisition of capacitance values has been terminated, such as when an operation for ending the currently active application is performed, for example, thecontroller15 ends all processing in the flowchart illustrated inFIG. 3.
[Action During Second Control Mode]
• Next, if thecontroller15 switches to the second control mode because a contactless operation mode file is attached to the activated application program, thecontroller15 advances the process to step S2 and improves the sensitivity of the capacitive sensors in thedisplay unit5 by raising the output gain of the capacitive sensors by a given amount, for example. Thecontroller15 then advances the process to step S3.
• In other words, since contactless operations are conducted in a state where an operating object is not made to contact thedisplay unit5, capacitance values detected by the capacitive sensors will be detected at lower values than those during direct touch operations. For this reason, upon switching to the second control mode, thecontroller15 attempts to improve the sensitivity of the capacitive sensors by raising the output gain of the capacitive sensors by a given amount as discussed above, for example.
• Thus, changes in capacitance values corresponding to contactless operations can be detected with high sensitivity, making it possible to accurately detect the contactless operation position, etc.
• Next, in step S3, thecontroller15 functions as theapplication execution controller24 and reads the contactless operation mode file attached to the activated application program. Then, thecontroller15 functions as thecontactless operation enabler25 and thecontactless operation detector20, loads the contactless operation mode file thus read into the contactless operationmode file memory17, and advances the process to step S4.
• The contactless operation mode file is a file of contactless operation mode data expressing capacitance map transition patterns for individual contactless operation modes used to specify execution of a desired process in the currently activated application program.
• For this reason, by loading the contactless operation mode file into the contactless operationmode file memory17, capacitance map transition patterns for individual contactless operation modes used to specify execution of a desired process in the currently activated application program become held in the contactless operationmode file memory17.
• In step S4, thecontroller15 functions as theread controller19 and thereby acquires the capacitance values of all capacitive sensors in thedisplay unit5 as a capacitance map, and then advances the process to step S5.
• In step S5, thecontroller15 compares the capacitance values in the acquired capacitance map to the direct touch threshold level indicated by the broken line inFIG. 4A which is stored in thethreshold level memory18. In so doing, thecontroller15 determines whether or not there exist capacitance values from among the acquired capacitance values that are equal to or greater than the direct touch threshold level.
• The existence of capacitance values equal to or greater than the direct touch threshold level among the capacitance values in the acquired capacitance map means that a direct touch operation has been performed by the user. For this reason, thecontroller15 advances the process to step S6 and conducts processing for direct touch operations.
• In other words, upon advancing the process to step S6, thecontroller15 functions as theread controller19 and compares the capacitance values in the acquired capacitance map to the direct touch threshold level stored in thethreshold level memory18. In so doing, thecontroller15 creates two-dimensional coordinate information for thedisplay unit5 corresponding to the position and number of direct touches by the user (i.e., the number of fingers in a direct touch operation), and advances the process to step S7.
• Thecontroller15 functions as thetouch driver21 and stores the two-dimensional coordinate information corresponding to the detected direct touch operation in the storage area for direct touch operations in theoperation history register16.
• Next, upon advancing the process to step S7, thecontroller15 functions as theoperation detector26 of thewindow manager23 and detects a direct touch operation mode (direct touch operation pattern) and a direct touch operation position on the basis of the two-dimensional coordinate information corresponding to the direct touch operation acquired when thecontroller15 functioned as thetouch driver21, as well as two-dimensional coordinate information obtained during past direct touch operations being stored in the storage area for direct touch operations in theoperation history register16. Thecontroller15 then advances the process to step S8.
• In step S8, thecontroller15 functions as theapplication execution controller24, controls execution of a process from among respective processes in the currently active application program that corresponds to the detected direct touch operation mode (direct touch operation pattern) and direct touch operation position, and advances the process to step S9.
• In step S9, thecontroller15 functions as theread controller19 as discussed above and determines whether or not an application program is currently running. In so doing, thecontroller15 determines whether or not to continue detecting capacitance values.
• Then, in cases where it is determined that acquisition of capacitance values has been terminated, such as when an operation for ending the currently active application is performed, for example, thecontroller15 ends all processing in the flowchart illustrated inFIG. 3.
• Meanwhile, in cases where it is determined to continue detecting capacitance values, such as when an application program is currently running, thecontroller15 returns the process to step S4 and once again acquires the capacitance values of all capacitive sensors in the display unit. While an application program is running, thecontroller15 controls repeated execution of action for acquiring the capacitance values of all capacitive sensors in this way at given time intervals.
• Meanwhile, in step S5, a lack of capacitance values equal to or greater than the direct touch threshold level among the capacitance values in the acquired capacitance map means there is a possibility that a contactless operation has been performed by the user. For this reason, thecontroller15 advances the process to step S10 in the case where the existence of capacitance values equal to or greater than the direct touch threshold level is not detected.
• In step S10, thecontroller15 functions as theread controller19 and compares the acquired capacitance values to the contactless threshold level indicated by the chain line inFIG. 4B which is stored in thethreshold level memory18. In so doing, thecontroller15 determines whether or not there exist capacitance values among the acquired capacitance values which are equal to or greater than the contactless threshold level.
• In the case where capacitance values equal to or greater than the contactless threshold level does not exist, thecontroller15 discards the acquired capacitance values by treating them as noise, returns the process to step S4 via step S9, and once again acquires the capacitance values of all capacitive sensors in thedisplay unit5.
• In contrast, the existence of capacitance values equal to or greater than the contactless threshold level means that a contactless operation like that illustrated inFIG. 4C, for example, has been performed by the user. For this reason, thecontroller15 advances the process to step S31 inFIG. 7.
• At this point, the respective processing in step S5 and step S10 will be once again explained in detail.
• Exemplary capacitance values for all capacitive sensors detected during a direct touch operation are illustrated inFIG. 5. During a direct touch operation, only the capacitance value of the sensor where the direct touch operation is centered and the capacitance values of the immediately adjacent sensors are detected as large capacitance values, as indicated by being enclosed in bold lines inFIG. 5.
• Exemplary capacitance values for all capacitive sensors detected during a contactless operation are illustrated inFIG. 6. During a contactless operation, capacitance values that have changed to intermediate values over a wider range than during a direct touch operation are detected, as indicated by being enclosed in bold lines inFIG. 6.
• In the case of the mobile phone in this embodiment, there are a total of two threshold levels: a direct touch threshold level indicated by the broken line inFIGS. 4A and 4B, and a contactless threshold level indicated by the chain line inFIGS. 4A and 4B, which is set at a lower level than the level of the direct touch threshold level.
• Upon acquiring all capacitance values from thedisplay unit5, in step S5 thecontroller15 compares the capacitance values of respective sensors in the acquired capacitance map to the direct touch threshold level indicated by the broken line inFIGS. 4A and 4B. Then, thecontroller15 determines whether or not there exist capacitance values among the capacitance values from all sensors in thedisplay unit5 which are equal to or greater than the direct touch threshold level.
• The case where capacitance values equal to or greater than the direct touch threshold level do exist as illustrated inFIG. 4A means that a direct touch operation is being conducted on thedisplay unit5 by the user. For this reason, thecontroller15 controls the storage of two-dimensional coordinate information corresponding to the detected direct touch operation in the storage area for direct touch operations in theoperation history register16.
• In contrast, in the case where is it determined that capacitance values equal to or greater than the direct touch threshold level do not exist among the capacitance values from all sensors in thedisplay unit5, thecontroller15 advances the process to step S10 and respectively compares the capacitance values of respective sensors in the acquired capacitance map to the contactless threshold level indicated by the chain line inFIGS. 4A and 4B. Then, in this step S10, thecontroller15 determines whether or not there exist capacitance values among the respective capacitance values detected by all sensors in thedisplay unit5 which are less than the direct touch threshold level but also equal to or greater than the contactless threshold level.
• The existence of sensors having capacitance values which are lower than the direct touch threshold level but also equal to or greater than the contactless threshold level means that a contactless operation is being conducted on thedisplay unit5 by the user. For this reason, thecontroller15 controls the storage of the acquired capacitance map in the storage area for contactless operations in theoperation history register16 and advances the process to step S31 of the flowchart inFIG. 7 hereinafter described.
• When thecontroller15 advances the process to step S31 of the flowchart inFIG. 7 due to determining in step S10 that there exist capacitance values among the capacitance values in the capacitance map acquired from thedisplay unit5 which are lower than the direct touch threshold level indicated by the broken line inFIG. 4B but also equal to or greater than the direct touch threshold level indicated by the chain line inFIG. 4B, thecontroller15 next functions as thecontactless operation detector20 and determines whether or not a given number or more capacitance maps are being stored in the storage area for contactless operations in the operation history register16 such that a contactless operation pattern) can be detected.
• Thecontroller15 then advances the process to step S32 in the case where it is determined that the given number or more capacitance maps are being stored, while advancing the process to step S9 of the flowchart inFIG. 3 in the case where it is determined that the given number or more capacitance maps are not being stored.
• In the case of the mobile phone in this embodiment, capacitance maps are acquired from thedisplay unit5 at given time intervals while an application program is running, as discussed above. For this reason, upon advancing the process to step S9 of the flowchart inFIG. 3 due to determining that the given number or more capacitance maps are not being stored, thecontroller15 determines whether or not to continue detecting capacitance maps by determining whether or not an application program is currently running.
• Then, in the case where it is determined that acquisition of the next capacitance map has terminated, such when an operation for ending the currently active application program is given, for example, thecontroller15 ends all processing in the respective flowcharts illustrated inFIGS. 3 and 7.
• In contrast, if thecontroller15 determines to continue detecting capacitance maps, thecontroller15 returns the process to step S4 and from thedisplay unit5 acquires a capacitance map expressing the capacitance values of all capacitive sensors in thedisplay unit5 as discussed earlier. Then, if there exist capacitance values among the capacitance values in the acquired capacitance map which are lower than the direct touch threshold level but also equal to or greater than the contactless threshold level, thecontroller15 controls storage of the capacitance map in the storage area for direct contactless operations in theoperation history register16.
• In this way, in the case where a number of capacitance maps enabling detection of a contactless operation pattern (contactless operation mode) or more are not being stored in the operation history register16 while an application program is running, thecontroller15 returns the process to step S4 via step S9 of the flowchart inFIG. 3, and once again acquires a capacitance map and controls its storage in the storage area for contactless operations in theoperation history register16. In so doing, capacitance maps corresponding to a contactless operation are successively stored in the storage area for contactless operations in theoperation history register16.
• At this point, in the case where the activated application program does support contactless operations, thecontroller15 is configured to load contactless operation mode data, which indicates capacitance map transition patterns for individual contactless operation modes of contactless operations used by the application program, into the contactless operationmode file memory17 as the contactless operation mode file, as discussed earlier.
• For this reason, upon advancing the process to step S32 due to determining that a number of capacitance maps enabling detection of a contactless operation mode are being stored in the storage area for contactless operations in theoperation history register16, thecontroller15 functions as thecontactless operation detector20 to specify for each capacitance map the placement positions on thedisplay unit5 of the capacitive sensors that detected capacitance values which are lower than the direct touch threshold level but also equal to or greater than the contactless threshold level, on the basis of the respective capacitance maps stored in theoperation history register16. Then, thecontroller15 detects shifts in the capacitive sensors specified for the individual capacitance maps as a capacitance map transition pattern corresponding to a contactless operation.
• An exemplary transition pattern across respective capacitance maps during a contactless operation is illustrated inFIGS. 8A to 8C. The example illustrated inFIGS. 8A to 8C is an example for the case where a contactless operation is made on thedisplay unit5 of a mobile phone from left to right along the short axis of thedisplay unit5.
• In the case where such a contactless operation is conducted, the positions of the capacitive sensors that detect capacitance values equal to or greater than the contactless threshold level move from left to right along the short axis of thedisplay unit5, as indicated by being respectively enclosed in bold lines inFIGS. 8A to 8C.
• In step S32, thecontroller15 detects the shift in the positions of the capacitive sensors that detect capacitance values equal to or greater than the contactless threshold level due to such a contactless operation as a capacitance map transition pattern corresponding to a contactless operation.
• Next, thecontroller15 advances the process to step S33 and functions as thecontactless operation detector20 to compare the capacitance map transition pattern detected with respect to a contactless operation in step S32 to capacitance map transition patterns expressed by contactless operation mode data in a contactless operation mode file loaded into the contactless operationmode file memory17, and specify a contactless operation mode (contactless operation gesture). Then, thecontroller15 detects two-dimensional coordinate information corresponding to the specified contactless operation.
• In other words, during a direct touch operation, capacitance values change over an extremely tight area on thedisplay unit5, as illustrated inFIG. 5. For this reason, two-dimensional coordinates on thedisplay unit5 corresponding to a direct touch operation can also be accurately detected.
• In contrast, during a contactless operation, capacitance values change over a wider area than during a direct touch operation, as illustrated inFIG. 6 andFIGS. 8A to 8C. For this reason, it is difficult to accurately detect two-dimensional coordinates on thedisplay unit5 corresponding to a contactless operation.
• For this reason, in the case of the mobile phone in this embodiment, contactless operation mode data in a contactless operation mode file is made up of data expressing contactless operation modes (gestures) corresponding to capacitance map transition patterns such as “right-to-left”, “left-to-right”, “top-to-bottom”, and “bottom-to-top”, as well as data expressing two-dimensional coordinates corresponding to contactless operation modes.
• Stated differently, since it is difficult to accurately detect two-dimensional coordinates on thedisplay unit5 corresponding to a contactless operation position, individual capacitance map transition patterns and individual contactless operation modes are associated with each other in advance, while in addition, two-dimensional coordinates for specifying processes in an application program are respectively associated with the individual contactless operation modes. The result is taken to be the contactless operation mode data.
• In step S33, thecontroller15 detects from the contactless operationmode file memory17 the contactless operation mode data of a transition pattern that matches the capacitance map transition pattern detected with respect to a contactless operation. As discussed above, the contactless operation mode data is made up of data expressing contactless operation modes corresponding to capacitance map transition patterns, and data expressing two-dimensional coordinates corresponding to the contactless operation modes for specifying processes of the application program.
• For this reason, by detecting from the contactless operationmode file memory17 the contactless operation mode data of a transition pattern that matches a capacitance map transition pattern detected with respect to a contactless operation, thecontroller15 detects a contactless operation mode and two-dimensional coordinates corresponding to this contactless operation mode.
• Next, thecontroller15 advances the process to step S34 and functions as theoperation detector26 to recognize a contactless operation mode and two-dimensional coordinate information corresponding to this contactless operation mode. Then, in step S34, thecontroller15 functions as theapplication execution controller24 to control execution of a process in an application program on the basis of the recognized contactless operation mode and two-dimensional coordinate information corresponding to this contactless operation mode.
• In so doing, execution of the process that corresponds to the user's contactless operation from among the various processes in the currently active application program is controlled by thecontroller15 on the basis of the currently active application program.
• Upon controlling execution of a process in an application program corresponding to a contactless operation in this way, thecontroller15 advances the process to step S9 inFIG. 3 and determines whether or not to continue detecting capacitance maps by determining whether or not an application program is currently running, as discussed earlier.
• Then, if thecontroller15 determines to continue detecting capacitance maps, thecontroller15 returns the process to step S4 and acquires from thedisplay unit5 a capacitance map expressing the capacitance values of all sensors on thedisplay unit5, as discussed earlier.
• In contrast, if thecontroller15 determines that acquisition of the next capacitance map has terminated, such when an operation for ending the currently active application program is given, for example, thecontroller15 ends all processing in the respective flowcharts illustrated inFIGS. 3 and 7.
[Advantages of Embodiment]
• As the foregoing explanation demonstrates, the mobile phone in this embodiment is provided with adisplay unit5 which forms a capacitive touch panel, and in addition, a direct touch threshold level as well as a contactless threshold level are set. Furthermore, two-dimensional coordinate information expressing a direct touch operation position is created on the basis of the capacitance values of the respective sensors in thedisplay unit5 during a direct touch operation and the direct touch threshold level, and the execution of information processing corresponding to the created two-dimensional coordinate information expressing a direct touch operation position from among various information processing in the currently active application program is controlled.
• Also, two-dimensional coordinate information expressing a contactless operation mode is created on the basis of the capacitance values of the respective sensors in thedisplay unit5 during a contactless operation and the contactless threshold level, and the execution of information processing corresponding to the created two-dimensional coordinate information expressing a contactless operation mode from among various information processing in the currently active application program is controlled.
• In so doing, information processing based on a given application program can be made operable by contactless operations, and new operational modes for application programs can be provided.
[Information Processing Action by Various Application Programs]
• Hereinafter, exemplary information processing action by various application programs in a mobile phone of this embodiment will be described.
[Information Processing Action by a Telephone Application Program](Information Processing Action According to a Contactless Operation)
• First, a mobile phone of this embodiment is configured to switch to what is called a hands-free telephony mode if a contactless operation is conducted so as to horizontally swipe a finger across thedisplay unit5 as illustrated inFIG. 9 in the case where an incoming telephone call or an incoming video phone call is received while the mobile phone is in a locked mode that cancels most operations.
• Specifically, if an incoming telephone call or an incoming video phone call is received while in the locked mode, thecontroller15 functions as thecontactless operation enabler25 to load a contactless operation mode file of contactless operation mode data expressing contactless operation modes (gestures) of contactless operations and two-dimensional coordinates corresponding to the contactless operation modes used by a telephone application program into the contactless operationmode file memory17.
• Also, thecontroller15 acquires the capacitance maps discussed earlier from thedisplay unit5 and controls their successive storage in theoperation history register16.
• Next, thecontroller15 compares the transition pattern of the capacitance maps stored in the operation history register16 to the capacitance map transition patterns for individual contactless operation modes expressed by the respective contactless operation mode files loaded into the contactless operationmode file memory17. Then, thecontroller15 detects contactless operation mode data expressing a transition pattern matching the transition pattern of the respective capacitance maps acquired from the operation history register16 from among the capacitance map transition patterns for individual contactless operation modes expressed by the respective contactless operation mode data stored in the contactless operationmode file memory17.
• The contactless operation mode data expresses a contactless operation pattern corresponding to a transition pattern of the respective capacitance maps, and two-dimensional coordinate information corresponding to this contactless operation pattern. Thecontroller15 functions as theoperation detector26 of thewindow manager23 to detect a contactless operation pattern and two-dimensional coordinate information corresponding to this contactless operation pattern on the basis of the detected contactless operation mode data.
• The detected contactless operation pattern is for a contactless operation instructing the switch to a hands-free telephony mode in the case of a contactless operation pattern wherein a finger is moved so as to horizontally swipe across thedisplay unit5. Thus, thecontroller15 functions as theapplication execution controller24 to switch to a hands-free telephony mode on the basis of a telephone application program, and controls driving of themicrophone unit4 which picks up telephone transmitter audio and thesecond speaker unit3b, which is an external speaker unit for obtaining the acoustic output of telephone receiver audio, while also controlling communication by thecommunication circuit2 so as to transmit and receive telephony audio obtained thereby.
• Similarly, in the case of detecting the above-discussed contactless operation wherein a finger is moved so as to horizontally swipe across thedisplay unit5 when an incoming video phone call is received, thecontroller15 switches to a hands-free telephony mode on the basis of a video phone application program, and controls driving of themicrophone unit4 which picks up telephone transmitter audio, thesecond speaker unit3b, which is an external speaker unit for obtaining the acoustic output of telephone receiver audio, thesecond camera unit8bfor obtaining transmitter images, and thedisplay unit5 for displaying receiver images, while also controlling communication by thecommunication circuit2 so as to transmit and receive telephony audio and images obtained thereby.
• Thus, when an incoming telephone call or an incoming video phone call is received, telephone or video phone communication in what is called a hands-free telephony mode can be made possible simply by conducting a contactless operation so as to horizontally swipe a finger across thedisplay unit5.
(Information Processing Action According to a Direct Touch Operation)
• Next, a mobile phone of this embodiment is configured such that if a direct touch operation is conducted so as to slide an unlock slider displayed on thedisplay unit5 in an unlock direction as illustrated inFIG. 9 in the case where an incoming telephone call is received while in the locked mode, the mobile phone recognizes that an off-hook operation has been performed, cancels the locked mode, and switches to a normal telephony mode that conducts telephony using thefirst speaker unit3aand themicrophone unit4.
• Specifically, if thecontroller15 detects a direct touch operation that slides the unlock slider in the unlock direction on the basis of a transition pattern of respective capacitance maps stored in the operation history register16 as discussed earlier in the case where an incoming telephone call is received while in the locked mode, thecontroller15 controls the driving of thefirst speaker unit3a, which is an internal speaker unit, and themicrophone unit4, and switches to a normal telephony mode.
• Thus, if a direct touch operation that slides an unlock slider in the unlock direction is conducted when an incoming telephone call is received, the lock is released and telephony can be conducted in a normal telephony state.
(Other Information Processing Action According to a Direct Touch Operation)
• Next, a mobile phone of this embodiment is configured such that if a direct touch operation is conducted so as to slide an unlock slider displayed on thedisplay unit5 in an unlock direction as illustrated inFIG. 9 in the case where an incoming video phone call is received while in the locked mode, the mobile phone cancels the locked mode and switches to a hands-free telephony mode.
• Specifically, if thecontroller15 detects a direct touch operation that slides an unlock slider displayed on thedisplay unit5 in an unlock direction on the basis of a transition pattern of respective capacitance maps stored in the operation history register16 as discussed earlier in the case where an incoming video phone call is received while in the locked mode, thecontroller15 cancels the locked mode while also switching to a hands-free telephony mode on the basis of a video phone application program.
• Then, thecontroller15 controls driving of themicrophone unit4 which picks up telephone transmitter audio, thesecond speaker unit3b, which is an external speaker unit for obtaining the acoustic output of telephone receiver audio, thesecond camera unit8bfor obtaining transmitter images, and thedisplay unit5 for displaying receiver images, while also controlling communication by thecommunication circuit2 so as to transmit and receive telephony audio and images obtained thereby.
• Thus, when an incoming video phone call is received, video phone communication in what is called a hands-free telephony mode can be made possible simply by conducting a direct touch operation that slides an unlock slider displayed on thedisplay unit5 in an unlock direction.
[Information Processing Action of an Email Management Program According to a Contactless Operation]
• Next, if an email is received, thecontroller15 activates an email management program stored in thememory14 and notifies the user of the receipt of the email.
• In this example, when the email management program is activated, a contactless operation that horizontally swipes a finger across thedisplay unit5 becomes a contactless operation instructing display of the body text of the received email.
• For this reason, when thecontroller15 detects a contactless operation that horizontally swipes a finger across thedisplay unit5 on the basis of the created two-dimensional coordinate information while functioning as thewindow manager23, thecontroller15 controls display of the body text of the received email on thedisplay unit5 in accordance with the contactless operation.
• Also, in this example, when the email management program is activated, a contactless operation that moves a finger up or down along the long axis of thedisplay unit5 becomes a contactless operation instructing scrolling of the body text of the displayed email.
• For this reason, when thecontroller15 detects a contactless operation that moves a finger over thedisplay unit5 along the long axis of thedisplay unit5 on the basis of the created two-dimensional coordinate information while functioning as thewindow manager23, thecontroller15 scrolls the body text of the email being displayed in accordance with the contactless operation.
• Thus, when an email is received, the body text of the received email can be displayed on thedisplay unit5 simply by conducting a contactless operation so as to horizontally swipe a finger across thedisplay unit5. Also, when the body text of an email is being displayed on thedisplay unit5, by simply conducting a contactless operation that moves a finger over thedisplay unit5 along the long axis of thedisplay unit5, the body text of the displayed email can be scrolled in accordance with the contactless operation along the long axis of thedisplay unit5.
[Information Processing Action of a Lock/Unlock Application Program According to a Contactless Operation]
• Next, if a givenhardware key6 is operated, thecontroller15 switches to a locked mode that cancels most key operations on the basis of a lock/unlock application program stored in thememory14.
• During such a locked mode, a contactless operation that horizontally swipes a finger across thedisplay unit5 becomes a contactless operation instructing the mobile phone to accept a key operation and switch from the locked mode to an unlocked mode which conducts information processing corresponding to the key operation.
• Also, during the unlocked mode, a contactless operation that horizontally swipes a finger across thedisplay unit5 becomes a contactless operation instructing the mobile phone to switch from the unlocked mode to a locked mode which cancels most key operations.
• Also, during the locked mode, a direct touch operation that slides an unlock slider on thedisplay unit5 in an unlock direction becomes a direct touch operation instructing the mobile phone to switch from the locked mode to the unlocked mode.
• Also, during the unlocked mode, a direct touch operation that slides an unlock slider on thedisplay unit5 in a lock direction becomes a direct touch operation instructing the mobile phone to switch from the unlocked mode to the locked mode.
• During the locked mode, if thecontroller15 detects a contactless operation that horizontally swipes a finger across thedisplay unit5 on the basis of the created two-dimensional coordinate information while functioning as thewindow manager23, thecontroller15 switches to the unlocked mode in accordance with the contactless operation, accepts a key operation by the user, and controls execution of information processing corresponding to the key operation.
• Also, during the unlocked mode, if thecontroller15 detects a contactless operation that horizontally swipes a finger across thedisplay unit5 on the basis of the created two-dimensional coordinate information while functioning as thewindow manager23, thecontroller15 switches to the locked mode and cancels most key operations.
• Thus, during a locked mode, the mobile phone can be switched to an unlocked mode simply by conducting a contactless operation that horizontally swipes a finger across thedisplay unit5, while during an unlocked mode, the mobile phone can be switched to a locked mode simply by conducting a contactless operation that horizontally swipes a finger across thedisplay unit5.
• Also, during the locked mode, if thecontroller15 detects a direct touch operation that slides an unlock slider on thedisplay unit5 in an unlock direction on the basis of the created two-dimensional coordinate information while functioning as thewindow manager23, thecontroller15 switches to an unlocked mode in accordance with the direct touch operation, accepts a key operation by the user, and controls execution of information processing corresponding to the key operation.
• Also, during the unlocked mode, if thecontroller15 detects a direct touch operation that slides an unlock slider on thedisplay unit5 in a lock direction on the basis of the created two-dimensional coordinate information while functioning as thewindow manager23, thecontroller15 switches to a locked mode and cancels most key operations.
• Thus, during a locked mode, the mobile phone can be switched to an unlocked mode simply by conducting a direct touch operation that slides an unlock slider on thedisplay unit5 in an unlock direction, while during an unlocked mode, the mobile phone can be switched to a locked mode simply by conducting a direct touch operation that slides an unlock slider on thedisplay unit5 in a lock direction.
[Information Processing Action of a Web Browsing Program According to a Contactless Operation]
• Next, in the case where a given web page is being displayed on thedisplay unit5 on the basis of a web browsing program stored in thememory14, a contactless operation that causes a finger to revolve in a clockwise direction over thedisplay unit5 becomes a contactless operation instructing the mobile phone to enlarge the web page being displayed.
• Also, in the above case where a web page is being displayed on thedisplay unit5, a contactless operation that causes a finger to revolve in a counter-clockwise direction over thedisplay unit5 becomes a contactless operation instructing the mobile phone to reduce the web page being displayed.
• Also, in the above case where a web page is being displayed on thedisplay unit5, a contactless operation that moves a finger over thedisplay unit5 upwards along the long axis of thedisplay unit5 becomes a contactless operation instructing the mobile phone to scroll up the web page being displayed.
• Also, in the above case where a web page is being displayed on thedisplay unit5, a contactless operation that moves a finger over thedisplay unit5 downwards along the long axis of thedisplay unit5 becomes a contactless operation instructing the mobile phone to scroll down the web page being displayed.
• For this reason, if thecontroller15 detects a contactless operation that causes a finger over thedisplay unit5 to revolve in a clockwise direction on the basis of the created two-dimensional coordinate information while functioning as thewindow manager23, thecontroller15 enlarges the web page being displayed at gradually larger display magnifications in accordance with the contactless operation.
• Also, if thecontroller15 detects a contactless operation that causes a finger over thedisplay unit5 to revolve in a counter-clockwise direction on the basis of the created two-dimensional coordinate information while functioning as thewindow manager23, thecontroller15 reduces the web page being displayed at gradually smaller display magnifications in accordance with the contactless operation.
• Also, if thecontroller15 detects a contactless operation that moves a finger over thedisplay unit5 upwards along the long axis of thedisplay unit5 on the basis of the created two-dimensional coordinate information while functioning as thewindow manager23, thecontroller15 scrolls up the web page being displayed in accordance with the contactless operation.
• Also, if thecontroller15 detects a contactless operation that moves a finger over thedisplay unit5 downwards along the long axis of thedisplay unit5 on the basis of the created two-dimensional coordinate information while functioning as thewindow manager23, thecontroller15 scrolls down the web page being displayed in accordance with the contactless operation.
• Thus, when a web page is being displayed, the web page can be enlarged or reduced by conducting a contactless operation that causes a finger over thedisplay unit5 to revolve in a clockwise or counter-clockwise direction.
• Also, when a web page is being displayed, the web page can be scrolled up or scrolled down by conducting a contactless operation that moves a finger over thedisplay unit5 upwards or downwards along the long axis of thedisplay unit5.
[Modifications]
• In the foregoing embodiment, the mobile phone is configured to detect both contactless operations and direct touch operations, but the mobile phone may also detect only contactless operations and control execution of information processing in application program on the basis of detected contactless operations. Even in such cases, the same advantages as discussed above can be obtained.
• Although the foregoing embodiment was an example wherein the present disclosure was applied to a mobile phone, the present disclosure may also be applied to other electronic devices besides a mobile phone, such as a PHS phone (PHS: Personal Handyphone System), a PDA (PDA: Personal Digital Assistant), a digital camera, a digital video camera, a portable game console, or a notebook computer, for example. Moreover, the same advantages as those of the foregoing embodiment can still be obtained in any case.
• Lastly, the foregoing embodiment is one example of the present disclosure. Accordingly, the present disclosure is not limited to the foregoing embodiment, and various modifications, combinations, and other embodiment may occur depending on design or other factors while remaining within the scope of the claims of the present disclosure or their equivalents. This is naturally understood by those skilled in the art.

Claims (19)

1. An information processing apparatus comprising:

a touch panel that detects a contact operation and a contactless operation of an operation element by detecting an electrostatic capacitance of an operation surface of the touch panel; and

a controller that

determines whether an application activated at the information processing apparatus supports contactless operations;

controls the application based only on contact operations detected at the touch panel when is determined that the application does not support contactless operations; and

controls the application based on both contact operations and contactless operations detected at the touch panel when it is determined that the application supports contactless operations.

2. The information processing apparatus ofclaim 1, wherein

the touch panel is a projection-type electrostatic-capacitance touch panel.

3. The information processing apparatus ofclaim 1, wherein

the touch panel includes a substrate layer including an insulator film, an electrode layer arranged below the insulator film, and a control integrated circuit.

4. The information processing apparatus ofclaim 3, wherein

the touch panel includes a plurality of mosaic electrode patterns each including two layers of transparent electrodes in each of vertical and horizontal directions in the electrode layer.

5. The information processing apparatus ofclaim 1, wherein

the controller increases a sensitivity of the touch panel when it is determined that the application supports contactless operations.

6. The information processing apparatus ofclaim 1, wherein

the controller receives electrostatic capacitance values output by the touch panel and compares the electrostatic capacitance values to a first predetermined threshold value.

7. The information processing apparatus ofclaim 6, wherein

the controller determines that the operation received at the touch panel is a contact operation when at least one of the electrostatic capacitance values is greater than or equal to the first predetermined threshold value.

8. The information processing apparatus ofclaim 6, wherein

the controller compares the electrostatic capacitance values to a second predetermined threshold value when none of the electrostatic capacitance values is greater than or equal to the first predetermined threshold value.

9. The information processing apparatus ofclaim 8, wherein

the controller determines that the operation received at the touch panel is a contactless operation when at least one of the electrostatic capacitance values is greater than or equal to the second predetermined threshold value.

10. The information processing apparatus ofclaim 6, wherein

the controller detects two-dimensional coordinate information corresponding to positions on the touch panel at which the electrostatic capacitance values are greater than or equal to the predetermined threshold value and controls execution of a process of the application based on the two dimensional coordinate information when is determined that the application does not support contactless operations.

11. The information processing apparatus ofclaim 6, wherein

the controller detects two-dimensional coordinate information corresponding to positions on the touch panel at which the electrostatic capacitance values are greater than or equal to the predetermined threshold value and controls execution of a process of the application based on the two dimensional coordinate information when is determined that the application supports contactless operations.

12. The information processing apparatus ofclaim 9, wherein

the controller detects a transition pattern of two-dimensional coordinate values corresponding to positions on the touch panel at which the electrostatic capacitance values are less than the first predetermined threshold and greater than or equal to the first predetermined threshold value and controls execution of a process of the application based on the transition pattern of two dimensional coordinate values when is determined that the application supports contactless operations.

13. The information processing apparatus ofclaim 1, wherein

the application is a telephone application that supports contactless operations,

the controller controls the telephone application to perform a hands-free mode off-hook operation when an incoming telephone call is received and a contactless operation is detected at the touch panel, and

the controller controls the telephone application to perform a normal-mode off-hook operation when an incoming call is received and a contact operation is detected at the touch panel.

14. The information processing apparatus ofclaim 1, wherein

the application is a videophone application that supports contactless operations, and

the controller controls the videophone application to perform a hands-free mode off-hook operation when an incoming videophone call is received and a contactless operation or a contact operation corresponding to an off-hook operation is detected at the touch panel.

15. The information processing apparatus ofclaim 1, wherein

the application is an electronic messaging application that supports contactless operations,

the controller controls the electronic messaging application to display a received electronic message when a first contactless operation is detected at the touch panel, and

the controller controls the electronic messaging application to scroll the displayed received electronic message when a second contactless operation is detected at the touch panel.

16. The information processing apparatus ofclaim 1, wherein

the application is a lock/unlock application that locks/unlocks access to the information processing apparatus that supports contactless operations, and

the controller controls the lock/unlock application to switch from a locked mode to an unlock mode when a contactless operation is detected at the touch panel.

17. The information processing apparatus ofclaim 1, wherein

the application is a display magnification application that controls a magnification level of information displayed by the information processing apparatus and that supports contactless operations, and

the controller controls increasing or decreasing a size of the information displayed by the information processing apparatus by a ratio corresponding to a contactless operation detected at the touch panel.

18. An information processing method performed by an information processing apparatus including a touch panel that detects a contact operation and a contactless operation of an operation element by detecting an electrostatic capacitance of an operation surface of the touch panel, the method comprising:

determining whether an application activated at the information processing apparatus supports contactless operations;

controlling the application based only on contact operations detected at the touch panel when is determined that the application does not support contactless operations; and

controlling the application based on both contact operations and contactless operations detected at the touch panel when it is determined that the application supports contactless operations.

19. A non-transitory computer-readable medium including computer program instructions, which when executed by an information processing apparatus including a touch panel that detects a contact operation and a contactless operation of an operation element by detecting an electrostatic capacitance of an operation surface of the touch panel, cause the information processing apparatus to perform a method, the method comprising:

determining whether an application activated at the information processing apparatus supports contactless operations;

controlling the application based only on contact operations detected at the touch panel when is determined that the application does not support contactless operations; and

controlling the application based on both contact operations and contactless operations detected at the touch panel when it is determined that the application supports contactless operations.

Images