US10595380B2 - Lighting wall control with virtual assistant - Google Patents

Abstract

A lighting wall controller can be a retrofit for existing light switches, and the lighting wall control replaces the conventional light switch and can still work as a conventional light switch or other power switch with “dumb” lights or appliances while providing the ability to control “smart” lights and/or other “smart” devices with voice commands. In addition to controlling lights and/or devices, voice commands can be used to provide information or actions back to the user in response to the voice command. The lighting wall controller thereby provides voice control functionality without requiring additional devices, such as additional voice control appliances.

Description

RELATED APPLICATIONS

This application claims the benefit of provisional patent application Ser. No. 62/400,525, filed Sep. 27, 2016, the disclosure of which is hereby incorporated herein by reference in its entirety.

FIELD OF THE DISCLOSURE

The present disclosure relates to wall controls for lighting systems, and in particular to lighting wall controls including extended functionality such as a voice-directed virtual assistant.

BACKGROUND

Networked “smart home” devices continue to grow in popularity, providing increasing levels of functionality and convenience. For example, traditional light bulbs and lighting fixtures are increasingly being replaced with light-emitting diode (LED) based bulbs and fixtures, which may be networked together in order to provide features such as remote control from a smart phone and basic automation. In addition, devices such as door locks, thermostats, connected power outlets, and media remote controls are now being network connected to add features beyond what has previously been possible. Due to the large variety of these devices, there is now an emerging market for home automation “hubs”, which are capable of communicating with a variety of these devices in order to provide a user with a single place to control all of their devices. While many of these home automation “hubs” accomplish this task, they are often discrete devices that must be separately added to a network. One type of home automation “hub” may provide voice control over one or more “smart home” devices. Referred to herein as a voice control appliance, these devices respond to voice commands by providing audible feedback or changing the settings of one or more “smart home” devices connected thereto. For example, the Amazon Echo is one such device that has gained popularity in recent years. While such devices may provide convenient “voice assistant” functionality, they are generally only capable of listening for voice commands in a relatively small space. That is, an installation may require several of these voice control appliances placed around a space in order to adequately hear voice commands issued by a user throughout the space. Providing voice control appliances in this manner may not only be unsightly, but may be impractical in some scenarios due to the fact that they generally require access to a power outlet which may not be available. Environmental obstructions may interfere with the ability of these voice control appliances to recognize voice commands due to the required placement of such a standalone device in a particular location.

Accordingly, there is a need for an improved way to communicate with networked “smart home” devices and distribute the control thereof within a space.

SUMMARY

The present disclosure relates to wall controls for lighting systems, and in particular to lighting wall controls including extended functionality such as a voice-directed virtual assistant. In one embodiment, a lighting wall controller can be a retrofit for existing light switches, and the lighting wall control replaces the conventional light switch and can still work as a conventional light switch or other power switch with “dumb” lights or appliances while providing the ability to control “smart” lights and/or other “smart” devices with voice commands. In addition to controlling lights and/or devices, voice commands can be used to provide information or actions back to the user in response to the voice command. The lighting wall controller thereby provides voice control functionality without requiring additional devices, such as additional voice control appliances.

In one embodiment, the lighting wall controller can include processing circuitry, a memory, and a user interface. The memory includes instructions, which, when executed by the processing circuitry cause the lighting wall controller to process a voice command received from a user via the user interface and perform one or more actions in response thereto.

In one embodiment, processing the voice command from the user includes transcribing the voice command and sending the transcribed voice command to a remote device. The remote device then determines one or more actions to be taken based on the transcribed voice command and sends the one or more actions back to the lighting wall controller. In response, the lighting wall controller executes the one or more actions.

In one embodiment, processing the voice command from the user includes sending the voice command or a processed version of the voice command to a remote device, where it is transcribed. The remote device then determines one or more actions to be taken based on the transcribed voice command and sends the one or more actions back to the wall controller. In response, the wall controller executes the one or more actions. For example, the user may want to know the answer to a question. The user can ask the question to the wall controller which sends the processed version of the voice command, (i.e., the question), to the remote device or voice control appliance, and the remote device or voice control appliance retrieves the answer to the question itself or through other devices. The remote device will send the answer to the question which will be transmitted to the user via a speaker, display or other user interface.

In one embodiment, the remote device is a device on the same local area network (LAN) as the lighting wall controller. There may be one or more intermediate devices through which the lighting wall controller communicates with the remote device over the LAN. In another embodiment, the remote device is a device located outside of the LAN of the lighting wall controller, for example, on a wide area network (WAN) to which the lighting wall controller connects to via a gateway. In one embodiment, the remote device is a voice control appliance. In another embodiment, the remote device is a server.

In one embodiment, processing the voice command from the user includes locally transcribing the voice command and determining one or more actions to be taken based on the transcribed voice command. In response, the lighting wall controller executes the one or more actions.

In some embodiments, the wall controllers form a network, such as a mesh network or partial (i.e., weak) mesh network, and transmit information or commands between each other. For example, a user in the master bedroom could send a command to turn off the lights in the kitchen. Depending on the embodiment, the voice command could go directly to the wall controller in the kitchen and the wall controller in the kitchen will turn off the lights, or the voice controller could go to a voice control appliance or other device which sends a command to the wall controller in the kitchen to turn off the lights.

In one embodiment, the one or more actions include controlling a light output of a light bulb and/or lighting fixture. In another embodiment, the one or more actions include displaying information for a user via the user interface.

Those skilled in the art will appreciate the scope of the present disclosure and realize additional aspects thereof after reading the following detailed description of the preferred embodiments in association with the accompanying drawing figures.

BRIEF DESCRIPTION OF THE DRAWING FIGURES

The accompanying drawing figures incorporated in and forming a part of this specification illustrate several aspects of the disclosure, and together with the description serve to explain the principles of the disclosure.

FIG. 1 is a diagram illustrating a lighting network according to one embodiment of the present disclosure.

FIG. 2 is a functional schematic illustrating a lighting wall controller according to one embodiment of the present disclosure.

FIG. 3 is a functional schematic illustrating a lighting wall controller according to one embodiment of the present disclosure.

FIG. 4 is a functional schematic illustrating a lighting wall controller according to one embodiment of the present disclosure.

FIG. 5 is a diagram of a user interface for a lighting wall controller according to one embodiment of the present disclosure.

FIG. 6 is a diagram of a user interface for a lighting wall controller according to one embodiment of the present disclosure.

FIG. 7 is a diagram of a user interface of a lighting wall controller according to one embodiment of the present disclosure.

FIG. 8 is a diagram of a user interface of a lighting wall controller according to one embodiment of the present disclosure.

FIG. 9 is a call-flow diagram illustrating communication between a lighting wall controller and a remote device according to one embodiment of the present disclosure.

FIG. 10 is a call-flow diagram illustrating communication between a lighting wall controller and a remote device according to one embodiment of the present disclosure.

FIG. 11 is a flow diagram illustrating a method of processing one or more voice commands according to one embodiment of the present disclosure.

DETAILED DESCRIPTION

The embodiments set forth below represent the necessary information to enable those skilled in the art to practice the embodiments and illustrate the best mode of practicing the embodiments. Upon reading the following description in light of the accompanying drawing figures, those skilled in the art will understand the concepts of the disclosure and will recognize applications of these concepts not particularly addressed herein. It should be understood that these concepts and applications fall within the scope of the disclosure and the accompanying claims.

It will be understood that, although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element, without departing from the scope of the present disclosure. As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items.

It will be understood that when an element such as a layer, region, or substrate is referred to as being “on” or extending “onto” another element, it can be directly on or extend directly onto the other element or intervening elements may also be present. In contrast, when an element is referred to as being “directly on” or extending “directly onto” another element, there are no intervening elements present. Likewise, it will be understood that when an element such as a layer, region, or substrate is referred to as being “over” or extending “over” another element, it can be directly over or extend directly over the other element or intervening elements may also be present. In contrast, when an element is referred to as being “directly over” or extending “directly over” another element, there are no intervening elements present. It will also be understood that when an element is referred to as being “connected” or “coupled” to another element, it can be directly connected or coupled to the other element or intervening elements may be present. In contrast, when an element is referred to as being “directly connected” or “directly coupled” to another element, there are no intervening elements present.

Relative terms such as “below” or “above” or “upper” or “lower” or “horizontal” or “vertical” may be used herein to describe a relationship of one element, layer, or region to another element, layer, or region as illustrated in the Figures. It will be understood that these terms and those discussed above are intended to encompass different orientations of the device in addition to the orientation depicted in the Figures.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises,” “comprising,” “includes,” and/or “including” when used herein specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. It will be further understood that terms used herein should be interpreted as having a meaning that is consistent with their meaning in the context of this specification and the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

As generally used herein, a “dumb” light or device is one that is simply controlled by adjusting or cutting of the power to the device, e.g. by a conventional light switch or TRIAC dimmer. A “smart” light or device is a device that includes decision making capability such that it can respond to signals, commands, feedback and/or information from sensors or other devices to adjust its operation.

FIG. 1 shows alighting network10 according to one embodiment of the present disclosure. Thelighting network10 includes a number of light bulbs12 (which, while not shown as such, may also be lighting fixtures without departing from the principles of the present disclosure), a number of lighting wall controls14, arouter16, agateway18, avoice control appliance20, avoice control server22, and aconnected device24, such as a smartphone, tablet, computer. Each one of these devices is connected to one another, either directly or via an intermediate device. These connections are illustrated inFIG. 1 as lines located between the devices, and may represent wired or wireless connections in any number of different communications technologies and/or protocols.

For example, each one of the lighting wall controls14 may include multiple communication interfaces as discussed below in order to communicate with thelight bulbs12 using a first communication technology and/or protocol, communicate with thesmartphone24 using a second communication technology and/or protocol, and communicate with thevoice control server22 using a third communication technology and/or protocol.

Together, thelight bulbs12, the lighting wall controls14, therouter16, thevoice control appliance20, thevoice control server22, and thesmartphone24 may form a local-area network (LAN). Communications between these devices may occur directly or through one or more intermediate devices such as therouter16, which may facilitate communications between all of the devices. Thegateway18 may connect the LAN to a wide-area network (WAN), such as the Internet. In some embodiments, thevoice control server22 may connect to the devices in thelighting network10 via the LAN. In other embodiments, thevoice control server22 connects to the devices in the lighting network via the WAN.

Thelight bulbs12 are configured to receive power, for example, from an alternating current (AC) line source along with one or more control signals and provide a light output based thereon. One or more of thelight bulbs12 may be “dumb” bulbs that are conventionally controlled, for example by an AC input signal AC_IN. Theselight bulbs12 generally provide a light output that is proportional to an average amount of energy provided by the AC input signal AC_IN (e.g., via a triode for alternating current (TRIAC) dimmer), and do not include a means for communicating with other devices. Otherlight bulbs12 may be “smart” bulbs equipped with electronics to provide decision making capabilities and communications circuitry such that they are capable of receiving data from other devices such as one or more of the lighting wall controls14 and adjusting the light output thereof based on the commands. In some embodiments, these “smart”light bulbs12 may also be controlled by conventional means as discussed above.

Each one of the lighting wall controls14 is configured to receive user input and power, for example, from an AC line source, and control a light output from one or more of thelight bulbs12 in response thereto. The lighting wall controls14 may do so by providing a user interface, which may be mechanical or software based (e.g., a touchscreen). To control the light output of thelight bulbs12, the lighting wall controls14 may provide the control signals thereto via a wired communications interface or a wireless communications interface. The wired control signals may be conventional alternating current (AC) dimmer signals (e.g., as provided by a dimmer switch such as a TRIAC dimmer), commands sent via an AC line interface (e.g., by modulating or otherwise transmitting data over the AC line), and/or Ethernet control signals. The wireless control signals may be Bluetooth, Zigbee, Thread, and/or Z-Wave control signals. In short, any type of wired or wireless control signals may be used to control a light output of thelight bulbs12, and the type of control signals used may be dependent on theindividual light bulbs12 themselves as discussed above.

In addition to the above, each one of the lighting wall controls14 may communicate among themselves in order to synchronize tasks, share sensor data, coordinate listening for or responding to voice commands from a user, or the like. In one embodiment, the lighting wall controls14 form a mesh network or a light mesh network in order to communicate with one another. Accordingly, the lighting wall controls14 may relay commands between one another, allowing voice commands or user input provided at one of the lighting wall controls14 to execute one or more actions on a differentlighting wall control14. For example, a voice command from a user may indicate that the user wishes to dim the lights in a particular location, such as the master bedroom. If the voice command is not received by alighting wall control14 located in the master bedroom, thelighting wall control14 may relay this command to the appropriatelighting wall control14, thereby allowing for the execution of the command.

To this end, each one of the lighting wall controls14 may be associated with a particular location in a space. For example, alighting wall control14 may be associated with a master bedroom, a kitchen, a conference room, or the like. These locations, which may be provided by a user, determined automatically, or some combination thereof, may allow a user to provide voice commands that are spatially oriented such as the example given above where a user wishes to dim the lights in a master bedroom. Such a voice command will be communicated as necessary to the appropriatelighting wall controller14 in order to execute the command. Associating the lighting wall controls14 with locations may be especially important when thelight bulbs12 connected thereto are conventionally controlled, since thelighting wall control14 is then the exclusive control point for the light output of these conventionally controlledlight bulbs12. When thelight bulbs12 include their own communications circuitry, interveninglighting wall controllers14 may be bypassed such that thelighting wall controller14 receiving a voice command may adjust the light output of thelight bulbs12 regardless of whether it is physically attached to them or located in the same room. In such scenarios, thelight bulbs12 themselves may be associated with a particular location in order to effectuate such behavior.

Notably, thelighting wall controllers14 may control other “smart” devices in addition to thelight bulbs12. For example, thelighting wall controllers14 may directly or indirectly provide commands to door locks, thermostats, media controllers, connected power outlets, and the like based on voice commands from a user as described in detail below.

In the embodiment shown inFIG. 1, the lighting wall controls14 act as a gateway for thelight bulbs12, connecting them to thelighting network10. However, in one embodiment a separate lighting gateway is provided through which thelight bulbs12 and the lighting wall controls14 connect to other devices in thelighting network10. In such an embodiment, the lighting wall controls14 may have a reduced number of communication interfaces in order to simplify the design thereof.

The control signals provided from the lighting wall controls14 to thelight bulbs12 may control any number of different parameters of the light provided therefrom. For example, the control signals from the lighting wall controls14 may cause thelight bulbs12 to change an intensity of a light provided therefrom, a color of the light provided therefrom, a color temperature of the light provided therefrom, a color rendering index of the light provided therefrom, or any other desired parameter.

Each of the lighting wall controls14 may control different groups oflight bulbs12 throughout thelighting network10. These groups oflight bulbs12 may be controlled via different communication interfaces as shown inFIG. 1. For example, the lighting wall controls14 may control the light output of a first group oflight bulbs12 via an AC interface, providing AC dimming signals thereto. Accordingly, the lighting wall controls14 may be connected to the first group oflight bulbs12 via an AC line. Further, the lighting wall controls14 may control the light output of a second group oflight bulbs12 via a wireless interface such as those discussed above. Accordingly, the lighting wall controls14 do not have to be connected to the second group oflight bulbs12 directly. The lighting wall controls14 may operate the first group oflight bulbs12 and the second group oflight bulbs12 in a dependent (i.e., synchronous) or independent manner. That is, the lighting wall controls14 may ensure that the light output from the first group oflight bulbs12 substantially matches that of the second group oflight bulbs12, or may operate thelight bulbs12 such that the light output from the first group oflight bulbs12 is different from that of the second group oflight bulbs12. In this way, the lighting wall controls14 may “bridge” the control of multiple groups oflight bulbs12 in thelighting network10, each of which may be operate via a different communications interface in order to provide seamless control oflight bulbs12 throughout a space. While the lighting wall controls14 are shown coupled toseparate light bulbs12 in thelighting network10, thelight bulbs12 controlled by each one of the lighting wall controls14 may overlap in some embodiments. As discussed above, lighting wall controls14 in thelighting network10 may receive user input or voice commands from users which require execution of actions on other lighting wall controls14. This may occur, for example, when changes to a light output oflight bulbs12 or group oflight bulbs12 exclusively controlled by a particularlighting wall control14 are requested by a user from a differentlighting wall controller14 or another device. This information may be passed to the appropriatelighting wall control14 as necessary to execute these actions as discussed above.

The lighting wall controls14 may receive commands from the connecteddevice24 such as a smartphone via a wired or wireless interface. As discussed above, the connecteddevice24 may be any suitable device such as a tablet, a smart watch, a dedicated remote control, or the like. In various embodiments, these commands may traverse one or more intermediate devices in thelighting network10 before reaching one or more of the lighting wall controls14. In response to these commands, one or more of the lighting wall controls14 may provide control signals to thelight bulbs12 in order to change a light output thereof.

In addition to the above, the lighting wall controls14 may receive commands from thevoice control appliance20 via a wired or wireless interface. As discussed above, thevoice control appliance20 is a standalone device for responding to voice commands from a user. Commands may be generated by thevoice control appliance20 in response to voice input from a user. In generating the commands, thevoice control appliance20 may interact with thevoice control server22. Thevoice control appliance20 and/orvoice control server22 may be configured to determine actions to take based on the voice commands from the user and relay these commands back to a requesting device. The computational complexity associated with natural language processing may necessitate the use of thevoice control server22 in some situations, since it may not be feasible to perform these computations on other devices in thelighting network10 that may have limited processing power and/or stringent efficiency requirements.

While thevoice control appliance20 may provide a convenient way to interact with one or more devices, alighting network10 may require several of them in order to adequately listen for voice commands within a given space. Since thevoice control appliance20 is a separate device dedicated only to that task, it may be expensive or inconvenient for a user to place a number of these throughout a space to provide the desired level of coverage. Generally, thesevoice control appliances20 recognize voice commands from a user in a relatively limited area. Accordingly, a substantial number of these devices must be placed strategically throughout a space in order to provide the desired functionality throughout the space. Further, these voice control appliances often require access to a power outlet, which may be problematic and/or produce unsightly results. The demands of these standalone devices may necessitate sub-optimal placement thereof such that the space in which voice commands are recognized is further reduced. Lighting wall controls14 such as the one shown inFIG. 1 may be located in every room of a space, and in some cases in more than one place in a room. Further, these lighting wall controls14 have access to power and are discreet in their appearance when compared to a dedicated device for which a user must find an appropriate spot. Finally, the placement of mostlighting wall controllers14 provides unrestricted access to sound waves in the surrounding area, and thus will be easily able to detect voice commands from a user. Accordingly, in order to provide voice control throughout the entirety of a space voice control or “virtual assistant” functionality is provided in the lighting wall controls14 as discussed below.

FIG. 2 shows details of alighting wall control14 according to one embodiment of the present disclosure. Thelighting wall control14 includesprocessing circuitry26, amemory28, auser interface30,communications circuitry32,sensor circuitry34, andpower management circuitry36. Theprocessing circuitry26 is configured to execute instructions stored in thememory28 in order to provide the primary intelligence of thelighting wall control14. In one embodiment, thememory28 includes avoice processing module38, which is a set of instructions stored in thememory28 configured to allow thelighting wall control14 to process voice commands as discussed below. While not shown, additional modules such as a fault detection module for detecting failures within thelighting wall control14, a diagnostic module for diagnosing said errors, and a protection module for security or other purposes may be provided as instructions stored in thememory28 or discrete circuitry in thelighting wall control14 to increase the robustness of the device.

Theuser interface30 allows a user to interact with thelighting wall control14, and may provide several ways to do so. For example, theuser interface30 may include a switch SW, which may be mechanical or any other type of switch, a capacitive or otherwise touch sensitive interface TCH, a display DSP, or the like. In some embodiments theuser interface30 may include a touchless interface (not shown), such as a three-dimensional gesture sensor, which may be provided using various sensors such as an image sensor. The display may be as simple or complex as desired. For example, the display may be an indicator LED, multiple indicator LEDs, an LED array, a full display such as a liquid crystal display (LCD), or any combination thereof. To provide the voice control capability discussed herein, theuser interface30 may include a microphone MIC and a speaker SPK. The microphone MIC may include multiple microphones, which may be provided in an array in order to more accurately recognize voice commands from a user. Further, the speaker SPK may include multiple speakers in order to provide better sound, or may connect to one or more remote speakers in order to provide audible feedback to a user.

Thecommunications circuitry32 may include multiple communications interfaces40, each of which may utilize a different communications technology and/or protocol to communicate with other devices in thelighting network10. For example, afirst communication interface40A may be a WiFi communications interface, asecond communication interface40B may be a Bluetooth communications interface, and an n^thcommunication interface40N may be a IEEE 802.15 communications interface. In short, thecommunications circuitry32 may include any number of different communications interfaces40 in order to communicate with a variety of devices in thelighting network10. As discussed above, in some embodiments thelighting wall control14 may include a limited number of communications interfaces40, and may communicate to other devices in thelighting network10 via a separate lighting gateway.

Thesensor circuitry34 may include any number of sensors to allow thelighting wall control14 to receive input from the surrounding environment. For example, thesensor circuitry34 may include an ambient light sensor ALS, an occupancy sensor OCC, and an image sensor IMG. The ambient light sensor ALS may provide a measurement of the ambient light in the surrounding environment to thelighting wall control14, which it may use to control a light output from one or more of thelight bulbs12. The occupancy sensor OCC may indicate whether or not the environment surrounding thelighting wall control14 is occupied by a person, which may be used by thelighting wall control14 to turn on and off thelight bulbs12. The image sensor IMG may be used to detect ambient light, occupancy, motion, and other light characteristics of thelight bulbs12. Any of these measurements may be used to adjust a light output of thelight bulbs12 in a desired fashion. Further, any number of additional sensor may be added to the sensor circuitry34 (e.g., temperature sensors, barometric pressure sensors, accelerometers, or the like) in order to allow thelighting wall control14 to collect additional information about the surrounding environment.

Thepower management circuitry36 may be configured to receive an AC input signal AC_IN, for example, an AC line voltage, and provide an AC output signal AC_OUT to one or more of thelight bulbs12. In doing so, thelighting wall control14 may dim or otherwise alter the light output of the light bulbs. In one embodiment, thepower management circuitry36 includes an AC dimmer (not shown). In other embodiments, thepower management circuitry36 includes power converter circuitry such as AC to direct current (DC) converter circuitry, power factor correction circuitry, rectifier circuitry, or the like (not shown). In some embodiments, thepower management circuitry36 may be configured to be wired in a three-way, four-way, or multiple-way AC circuit. Thepower management circuitry36 may cooperate with theprocessing circuitry26 in order to properly respond to AC signals received from other switches in the multiple-way configuration and to properly provide AC signals to other switches in the multiple-way configuration in order for all of the switches in the circuit to properly function. Where multiple switches in the circuit are lighting wall controls14 including intelligence such as the one discussed herein, the lighting wall controls14 may effectuate the multiple-way behavior by communicating in a wired or wireless manner. Where some of the switches in the circuit are “dumb” switches, thelighting wall control14 may manipulate an AC output thereof in order to effectuate the multiple-way behavior. Thelighting wall control14 may require pass-through or constant AC power to provide all of the functionality thereof, and such considerations must therefore be taken when including the lighting wall control in a multiple-way circuit. In addition to receiving AC input signals AC_IN, thepower management circuitry36 may also be configured to receive DC input signals, condition or otherwise alter these signals as desired, and provide one or more output signals to thelight bulbs12 to control the light output thereof. In some embodiments, thepower management circuitry36 may include a battery to provide power in the event of a power outage, or to ensure storage of settings or otherwise operate one or more aspects of thelighting wall control14 when line power is not available.

FIG. 3 shows alighting wall control14 according to an additional embodiment of the present disclosure. Thelighting wall control14 shown inFIG. 3 is substantially similar to that shown inFIG. 2, but further includes dedicatedvoice processing circuitry42 therein. The dedicatedvoice processing circuitry42 may be optimized for recognizing human speech. In one embodiment, the dedicatedvoice processing circuitry42 is configured to transcribe spoken words into text, data, or any appropriate form, which may then be parsed to determine one or more actions to be taken based thereon. Further, the dedicatedvoice processing circuitry42 may be optimized to listen for a “trigger phrase”, which may indicate that a person is providing a voice command to thelighting wall control14. Listening for a trigger phrase may prevent thelighting wall control14 from recording all spoken words in the surrounding environment in order to increase the privacy of users. To reduce the power consumption of thevoice processing circuitry42 and therefore optimize efficiency, it may be provided as a specialized application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or the like. Providing dedicatedvoice processing circuitry42 in thelighting wall control14 may free up valuable processing power in theprocessing circuitry26 for performing other tasks.

FIG. 4 shows alighting wall controller14 according to an additional embodiment of the present disclosure. For reference, a number oflight bulbs12 are also shown. Thelighting wall controller14 includes processing circuitry, which is configured to execute instructions stored in a memory to provide the central intelligence of thelighting wall controller14. A power supply module, along with a battery connected thereto, receives an AC input signal AC_IN and provides power to the processing circuitry, which may be distributed to other portions of the device. Fault and protection circuitry increases the robustness of the lighting wall controller by detecting faults and responding thereto. Wireless communications circuitry allows the lighting wall controller to communicate with other devices in thelighting network10. Gate drive circuitry controls a power device, which is in line with the AC line voltage provided to thelight bulbs12 in order to control the light output thereof in a conventional manner as discussed above. The power device may be a transistor device or any other suitable device for controlling the amount of energy delivered to thelight bulbs12. An indicator such as an LED or an LCD is provided, and may be used to provide feedback to a user as discussed above. A speaker and associated circuitry may similarly be used to provide audible feedback to a user. A voice recognition module along with a microphone attached thereto allows thelighting wall control14 to receive and respond to voice commands. A mechanical switch allows a user to cut power to thelight bulbs12 when desired. A number of sensors including an occupancy sensor, an ambient light sensor, an image sensor, a touch sensor (which may be a capacitive touch sensor), and a three-dimensional gesture sensor allow thelighting wall control14 to receive input from the surrounding environment. The processing circuitry is coupled to each one of the fault and protection circuitry, the wireless communications circuitry, the gate drive circuitry, the indicator, the speaker, the voice recognition module, the mechanical switch, and the sensors. Accordingly, the processing circuitry may receive input from these portions of the device or provide commands thereto to direct the activity of thelighting wall control14. In various embodiments, the processing circuitry may be a microcontroller unit or the like.

FIG. 5 shows auser interface30 for alighting wall control14 according to one embodiment of the present disclosure. As shown, theuser interface30 includes atouch panel44, which may be mechanical, capacitive, or otherwise touch sensitive. Further, while referred to as a “touch” panel, thetouch panel44 may respond to non-touch gestures such as those performed by a user in the space surrounding thelighting wall control14. Thetouch panel44 may control the intensity of light provided bylight bulbs12 controlled by the lighting wall controller based on input from a user. Afaceplate46 is provided around thetouch panel44. Thefaceplate46 may include afirst opening48, asecond opening50, and athird opening52. Thefirst opening48 may provide the microphone MIC access to the surrounding environment so that voice commands from a user may be detected. Thesecond opening50 may provide optical access to the surrounding environment for one or more of the ambient light sensor ALS, the occupancy sensor OCC, and the image sensor IMG. Additional openings may be provided in embodiments in which more than one of these sensors is provided. Thethird opening52 may provide the speaker SPK access to the surrounding environment so that audible feedback and other sounds may be provided from thelighting wall control14.

FIG. 6 shows auser interface30 for alighting wall control14 according to an additional embodiment of the present disclosure. Theuser interface30 is substantially similar to that shown inFIG. 5, except that thetouch panel44 shown inFIG. 5 is replaced with atouchscreen54. The touchscreen may display information about thelight bulbs12 controlled by thelighting wall control14 as well as any other devices in thelighting network10. For example, thetouchscreen54 may display the current occupancy status and the current brightness setting of thelight bulbs12 as shown inFIG. 6. Controls that are often used may be displayed in a prominent manner to allow a user to easily and intuitively control a light output of thelight bulbs12 connected to thelighting wall control14. An indicator may be provided that thelighting wall control14 is currently ready for voice commands from a user.

FIG. 7 shows theuser interface30 illustrated inFIG. 6 after a voice command has been detected by a user. Thetouchscreen54 may indicate the command that was detected and indicate an action that is currently being executed in response thereto. In some embodiments, a progress indicator may be provided. Further, feedback may be solicited to refine the accuracy of voice recognition of thelighting wall control14. For example, a prompt on the screen may ask whether the detected voice command was accurately transcribed, and whether the resulting action was the intended consequence of the detected voice command. After receiving this feedback, thelighting wall control14 and/or a backend device used for responding to the voice commands may alter the transcription and/or response to the voice commands in order to better respond to voice commands over time.

FIG. 8 shows theuser interface30 illustrated inFIG. 6 after a different type of voice command has been detected by a user. While voice commands may be used to instruct thelighting wall control14 to provide control signals to one or more other devices, they may also be used to request information from thelighting wall control14, which must then be displayed or otherwise communicated to the user. For example, a user may ask for the weather forecast, which may then be displayed as shown inFIG. 8. Other types of information may be requested and displayed as well. In various embodiments, audible feedback may be provided by the user in addition to displaying the information on thetouchscreen54 or other user interface. Such audible feedback may include computer generated speech responding to the request from the user.

FIG. 9 is a call flow diagram illustrating communications between alighting wall control14 and aremote device56 in order to execute one or more actions based on voice commands from a user according to one embodiment of the present disclosure. First, a voice command is received from a user (100). To receive a voice command, thelighting wall control14 may constantly listen for voice commands and/or trigger phrases via the microphone MIC as discussed above. The voice command may then be transcribed into text, data representative of the voice command, or any appropriate form (102). The voice transcription may be accomplished via theprocessing circuitry26 or the dedicatedvoice processing circuitry42. The transcribed voice command is then sent to a remote device56 (104), which may be thevoice control appliance20, thevoice control server22, or any other device, through one or more intermediate devices (e.g., therouter16, thegateway18, or any other device). Theremote device56 determines any necessary actions to be taken based on the transcribed voice command (106). For example, theremote device56 may use natural language processing along with machine learning algorithms to determine the intent of the voice command and how to respond. These actions are then sent back to the lighting wall control14 (108), where they are executed thereby (110).

As discussed above, the actions may include changing a light output of one or more of thelight bulbs12, displaying information, controlling one or more other devices in thelighting network10, or any other task. For example, a user may request thelighting wall control14 to “Turn on the lights,” to “Set the brightness of the lights inconference room 1 to 80%,” or to “Turn on the projector.” Thelighting wall control14 along with the remote device will determine the necessary actions to be taken based on these requests.

FIG. 10 is a call flow diagram illustrating communications between alighting wall control14 and aremote device56 in order to execute one or more actions based on voice commands from a user according to an additional embodiment of the present disclosure. First, a voice command is received from a user (200). The voice command is then sent to a remote device56 (202). As discussed above, theremote device56 may be thevoice control appliance20, thevoice control server22, or any other device, and communication with the remote device may occur between one or more intermediate devices. Sending the voice command to theremote device56 may include performing analog-to-digital conversion of the voice command from the user and sending a digital version thereof to theremote device56. In some embodiments, compression may be applied to the digital version of the voice command in order to reduce the required bandwidth of communication between thelighting wall control14 and theremote device56. Theremote device56 may then transcribe the voice command (204) using dedicated hardware or software as discussed above, and may determine any necessary actions to be taken based on the transcribed voice command (206) as discussed above. These actions are then sent back to the lighting wall control14 (208), where they are executed thereby (210).

FIG. 11 is a flow diagram illustrating a method for responding to voice commands from thelighting wall control14 according to one embodiment of the present disclosure. First, a voice command is received (300). The voice command is then transcribed into text, data representative of the voice command, or any appropriate form (302) for further processing. Necessary actions based on the transcribed voice command are then determined (304), and executed (306) by thelighting wall control14. Notably, the transcription and determination of actions based on the transcribed voice command are performed locally on thelighting wall control14 in the embodiment shown inFIG. 11. This may be enabled by the dedicatedvoice processing circuitry42 discussed above.

The above approaches inFIGS. 9-11 illustrate different ways that thelighting wall control14 could cooperate with a remote device such as thevoice control appliance20 and avoice control server22 in order to respond to voice commands from a user. In particular, they illustrate different ways to distribute the transcription and processing of voice commands from a user to accomplish a desired task based thereon. The above approaches illustrate that thevoice processing module38 and/or thevoice processing circuitry42 in thelighting wall control14 may perform several different levels of voice processing based on the embodiment. For example, the voice processing performed by thevoice processing module38 and/or thevoice processing circuitry42 may be a simple analog-to-digital conversion, or may involve more intensive processing such as voice-to-text transcription. In certain applications, thevoice processing module38 and/or thevoice processing circuitry42 may work alongside theprocessing circuitry26 in order to perform even more intensive processing such as natural language processing and the like in order to determine a desired action to be performed based on the voice commands. In short, the term “voice processing” used throughout the present application may indicate many different levels of intensity of processing of the voice commands.

Notably, the above are only exemplary approaches to such a problem. There are any numbers of ways in which a lighting wall controller could parse and respond to voice commands from a user, all of which are contemplated herein. Regardless of the details of how it is accomplished, providing hardware and accompanying software for detecting voice commands in alighting wall control14 allows voice command (i.e., “virtual assistant”) functionality to be distributed throughout a space without the need for a multitude of dedicated hardware that may be expensive or unsightly. That is, due to the fact that lighting wall controls14 are already integrated into a power infrastructure and distributed spatially throughout a home, these lighting wall controls14 offer significant benefits for providing an interface for voice control over dedicated hardware.

Those skilled in the art will recognize improvements and modifications to the preferred embodiments of the present disclosure. For example, this disclosure has focused on a lighting wall controller, but depending on the embodiment, the wall controller according to principles of the present disclosure need not control lights (or at least not in the conventional fashion) even though it replaces a conventional light switch or is mounted where a conventional light switch would typically be located. Additionally, the wall controllers can network with each other in various network structures, including with other devices, lights and/or sensors. All such improvements and modifications are considered within the scope of the concepts disclosed herein and the claims that follow.

Claims (24)

What is claimed is:

1. A lighting wall control comprising:

a user interface;

a voice processing module configured to process voice commands from a user;

power management circuitry configured to receive an AC input signal and provide an AC output signal suitable for powering one or more lights and controlling a light output thereof;

a communication interface configured to communicate with one or more additional lights; and

processing circuitry coupled to the user interface, the voice processing module, the power management circuitry, and the communication interface, the processing circuitry configured to:

adjust a light output of the one or more lights via the power management circuitry based on user input from the user interface and the voice commands processed by the voice processing module; and

adjust a light output of the one or more additional lights based on the user input from the user interface and the voice commands from the voice processing module.

2. The lighting wall control ofclaim 1 wherein the communication interface is a wireless communication interface.

3. The lighting wall control ofclaim 1 wherein the processing circuitry is configured to control the light output of the one or more lights and the light output of the one or more additional lights in a synchronous manner.

4. The lighting wall control ofclaim 1 wherein the processing circuitry is configured to control the light output of the one or more lights independently.

5. The lighting wall control ofclaim 1 wherein the processing circuitry is configured to:

receive commands from one or more other lighting wall controls via the communication interface, wherein the commands are generated from the one or more other lighting wall controls based on one of user input from a user interface and voice commands from a voice processing module; and

adjust the light output of the one or more lights based on the commands.

6. The lighting wall control ofclaim 1 wherein the lighting wall control is configured to:

receive the voice commands via a microphone;

transcribe the voice commands; and

determine one or more actions to be performed based on the transcribed voice commands.

7. The lighting wall control ofclaim 6 wherein the voice processing module is configured to process the voice commands by transcribing the voice commands.

8. The lighting wall control ofclaim 7 wherein the processing circuitry is configured to determine the one or more actions to be performed based on the transcribed voice commands.

9. The lighting wall control ofclaim 1 wherein the processing circuitry is further configured to:

transmit the voice commands to a remote server via the communication interface; and

receive one or more actions to be performed based on the voice commands from the remote server via the communication interface.

10. The lighting wall control ofclaim 9 wherein the voice processing module is configured to process the voice commands by performing an analog-to-digital conversion on the voice commands such that the voice commands transmitted to the remote server are transmitted in a digital format.

11. The lighting wall control ofclaim 9 wherein the processing circuitry is further configured to perform the one or more actions.

12. The lighting wall control ofclaim 1 wherein:

the voice processing module is configured to process the voice commands by transcribing the voice commands;

the processing circuitry is configured to transmit the transcribed voice commands to a remote server via the communication interface; and

the processing circuitry is configured to receive one or more actions to be performed based on the transcribed voice commands from the server via the communication interface.

13. The lighting wall control ofclaim 1 wherein the processing circuitry is configured to display visual information via the user interface in response to the voice commands.

14. The lighting wall control ofclaim 1 wherein the processing circuitry is further configured to request information from a remote server via the communication interface in response to the voice commands.

15. The lighting wall control ofclaim 1 wherein the processing circuitry is further configured to send a command to one or more other lighting wall controls via the communication interface based on the voice commands.

16. The lighting wall control ofclaim 1 further comprising a first communication interface and a second communication interface, wherein the processing circuitry is coupled to the first communication interface and the second communication interface and configured to:

transmit the voice commands to a remote server via the first communication interface;

receive one or more actions to be performed based on the voice commands from the remote server via the first communication interface; and

adjust the light output of the one or more lights via the second communication interface.

17. The lighting wall control ofclaim 16 wherein the voice processing module is configured to process the voice commands by performing an analog-to-digital conversion on the voice commands such that the voice commands transmitted to the remote server are transmitted in a digital format.

18. The lighting wall control ofclaim 16 wherein the first communication interface is configured to communicate with the remote server via a wide area network (WAN).

19. A lighting wall control comprising:

a user interface;

a voice processing module configured to process voice commands from a user;

power management circuitry configured to receive an AC input signal and provide an AC output signal suitable for powering one or more lights and controlling a light output thereof;

a communication interface configured to communicate with one or more additional lights; and

processing circuitry coupled to the user interface, the voice processing module, the power management circuitry, and the communication interface, the processing circuitry configured to:

adjust a light output of the one or more lights via the power management circuitry based on user input from the user interface and the voice commands processed by the voice processing module;

receive commands from one or more other lighting wall controls via the communication interface, wherein the commands are generated from the one or more other lighting wall controls based on one of user input from a user interface and voice commands from a voice processing module; and

adjust the light output of the one or more lights based on the commands from the one or more other lighting wall controls.

20. A lighting wall control comprising:

a user interface;

a voice processing module configured to process voice commands from a user;

a communication interface; and

processing circuitry coupled to the user interface, the voice processing module, and the communication interface, the processing circuitry configured to:

adjust a light output of one or more lights based on user input from the user interface and the voice commands processed by the voice processing module;

transmit the voice commands to a remote server via the communication interface; and

receive one or more actions to be performed based on the voice commands from the remote server via the communication interface.

21. The lighting wall control ofclaim 20 wherein the processing circuitry is further configured to perform the one or more actions.

22. A lighting wall control comprising:

a user interface;

a voice processing module configured to process voice commands from a user;

a communication interface; and

processing circuitry coupled to the user interface, the voice processing module, and the communication interface, the processing circuitry configured to:

adjust a light output of one or more lights based on user input from the user interface and the voice commands processed by the voice processing module; and

request information from a remote server via the communication interface in response to the voice commands.

23. A lighting wall control comprising:

a user interface;

a voice processing module configured to process voice commands from a user;

a communication interface; and

processing circuitry coupled to the user interface, the voice processing module, and the communication interface, the processing circuitry configured to:

adjust a light output of one or more lights based on user input from the user interface and the voice commands processed by the voice processing module; and

send a command to one or more other lighting wall controls via the communication interface based on the voice commands.

24. A lighting wall control comprising:

a user interface;

a voice processing module configured to process voice commands from a user;

a first communication interface;

a second communication interface; and

processing circuitry coupled to the user interface, the voice processing module, and the first and second communication interfaces, the processing circuitry configured to:

adjust a light output of one or more lights based on user input from the user interface and the voice commands processed by the voice processing module;

transmit the voice commands to a remote server via the first communication interface;

receive one or more actions to be performed based on the voice commands from the remote server via the first communication interface; and

adjust the light output of the one or more lights via the second communication interface.

Images