1	createSession ( )	Authenticate requester.
	destroySession ( )	Cancel authentication.
2	requestAccess (manifest)	Ask for permission to access
		certain devices.
3	singleQuery (name, params)	Get a single sensor sample.
	continuousQuery (name,	Start periodic fetch of sensor
	params, period)	samples.
4	startSensor (name)	Turn on sensor.
	stopSensor (name)	Turn off sensor.
	sensorAdded (name)	Upcall fired when a sensor is
		added.
	sensorRemoved (name)	Upcall fired when a sensor is
		removed.
	getSensorList ( )	Get available sensors.
5	enqueueKernel (kernel)	Queue a computation kernel
		for execution.
	setKernelData (parameters)	Set the input data for the computation
		pipeline.
	executeKernels ( )	Run the queued kernels on the input
		data.
6	put (storename, key, value)	Put value by key.
	get (storename, key)	Get value by key.

Sensor API

The API may control several device types316. To provide access tosensors318 like cameras, accelerometers, and GPS units and the like, the API provides a single-

sample query interface

324,344 and a

continuous query interface

324,330 as shown in the third set of calls in Table 1 and inFIG. 3. The API calls singleQuery (name, params) and continuous Query (name, params, period) accept an application-defined callback which is invoked when the data has arrived from the sensor. The data, for example, may be picture information if the sensor is a camera or position information if the sensor is a GPS receiver. The singleQuery (name, params) call accepts the name of the sensor device to query and a device-specific params value which controls sensor-specific properties such as audio sampling rate of data from a microphone. The continuous Query (name, params, period) call also accepts the name of the sensor device to query and a device-specific params value, and also includes an additional parameter representing the period of the sampling rate frequency for continuous query calls when the continuous query is started330.

The fourth set of calls in Table 1 and shown inFIG. 3 provide a sensor management interface. The API calls startSensor (name) and stopSensor (name) provide power controls that allow a web page to turn on or shut off devices that is not needed. The hardware device server ensures that a device is left on if at least one application still needs it. The API calls sensorAdded (name) and sensorRemoved (name) let applications register callbacks which provide notification to the application when devices arrive or leave. These events are common for off-platform devices such as Bluetooth headsets and some shoe sensors that track motion and distance traveled of the individual wearer. Inserting and removing USB port devices may also cause callbacks to provide notification of this status in some applications. The API call getSensorList ( ) produces a list of all available sensors available to a hardware device server.

Processor API

Multi-core processors andprogrammable GPUs320 are available on desktops and transitioning to mobile devices. Simple multi-processor computing models are available for exporting to mobile devices. A kernel represents a computational task to run on a core. Kernels are restricted to executing two types of predefined functions, primitive functions and built-in functions. Primitive functions are geometric, trigonometric, and comparator operations. Built-in functions are higher-level functions that have been identified as being particularly useful for processing hardware data, and may include the use of primitive functions. For example, built-in functions may include Fast Fourier Transform signal transforms and matrix operations.

Considering the fifth set of calls in Table 1, a web page passes a kernel to the API by calling enqueueKernel (kernel),326 as shown inFIG. 3. To execute a parallel vector computation with that kernel, the web page calls setKernelData (parameters) withvector arguments332. A new copy of the kernel will be instantiated for each argument, the kernels being run in parallel. A web page can also create a computational pipeline by calling enqueueKernel (kernel) multiple times with the same or different kernel. The kernels inputs and outputs will be chained in the order that the kernels were passed to enqueueKernel (kernel). The web page sets the input data for the pipeline by passing a scalar value to setKernelData (parameters). Once an application has created its kernels and set their input data, it calls executeKernels ( ) to start the computation to run the queued kernels on theinput data336. The kernels are distributed to various cores in the system, cross-kernel communication is coordinated, and an application-provided callback is executed when the computation is complete342.

Storage API

The sixth and final set of API calls in Table 1 andFIG. 3 provide a key/value storage interface322. The namespace is partitioned by web domain and by storage device. A web domain can only access data that resides in its partitions. To support removable devices, connection and disconnection notification events are provided similar to off platform sensors. The API call put (storename, key, value) stores a value by key in

storename

328,346, and get (storename, key) accesses a value by key instorename338.

Authenticating Hardware Requests

Privilege separation is used to provide a web page with hardware access while reducing security vulnerabilities. It lets parts of an application run with different levels of privilege, which may be used to prevent non-privileged or low-privileged parts of an application from gaining unauthorized access to parts with higher privileges. The web page and an enclosing web browser that executes the page's code are both considered to be untrusted. The web browser is placed in a sandbox which limits its access to certain services, including preventing direct access to hardware devices controlled by the hardware device server. The small, native code hardware device server resides in a separate process from the web browser, and executes hardware requests on behalf of the device protocol translator embedded within the web page application and browser, exchanging data with the device protocol translator via HTTP protocol and HTTP commands. The device protocol translator defines and implements the API described above. It provides for authentication purposes and translates page-initiated hardware requests into HTTP fetches. The hardware device server requires three housekeeping tasks to occur prior to enabling access to the hardware devices. These tasks include manifest authorization, session establishment and session teardown. It should be noted that remote or off-platform devices (see120 inFIG. 1) are seamlessly accessed in the same manner as the local devices.

If the manifest is authorized450, the hardware device server sends an authentication request to the requestingdomain435. Any subsequent request by the requesting web page must be authenticated by comparing the received authentication with a previously stored correspondingauthentication470. Authentication must be received445 before a session may be established455.

Before a web page can open a session and initiate hardware requests to the device protocol translator, it must get a new authentication from thehardware device server435. Once a session is established455, each request must be authenticated465 by comparing the received authentication with a corresponding authentication stored inmemory470. If a mapping does not exist between the received authentication and that stored in memory, the request is ignored by the

hardware device server

470,485. Authentication identifies a requestor as being legitimate for every request made while the session is open. If a mapping does exist between the received authentication and that stored inmemory470, the request is validated and the appropriate hardware device is accessed475. If additional requests are made480, anotherauthentication465 is required. Upon receipt of a request to terminate thesession480, access to the selected hardware device is terminated and the session is ended485.

Since the hardware device server receives requests expressed as HTTP fetches, a natural way for the hardware device server to authenticate requester identification is to inspect the referrer field in the HTTP request. This is a standard HTTP field which indicates the URL, and thus the domain, of the page that generated the request. Unfortunately, a misbehaving browser can subvert this authentication scheme by examining which domains successfully receive hardware data, and then generating fake requests containing these snooped referrer fields. This is essentially a replay attack on a weak authenticator.

To prevent these replay attacks, the hardware device server grants authentication to each authorized domain to prevent access by unauthorized domains. Before a page in a trusted domain can access the hardware device server, it must send asession establishment message415 to the hardware device server. The hardware device server examines the referrer of the HTTP message and checks whether the domain has already been granted authentication. If not, the server generates an authentication, stores the mapping between the domain and that authentication, and sends the authentication to the page. Later, when the page sends an actual hardware request to establish asession455, it includes the authentication in its HTTP request. If the authentication does not match the mapping found in the hardware device server's table470, the hardware device server ignores thehardware request485. If the authentication does match470, a session may be established455 and access to sensors, processors and storage devices is enabled475.

A page sends a session tear down message to thehardware device server480 when it no longer needs to access hardware attached to the hardware device server. Upon receipt of the tear downmessage480, the server deletes the relevant domain/token mapping and the session ends485. The device protocol translator detects when a page is about to unload by registering a handler for the device protocol translator unload event.

Given the capability scheme, a misbehaving browser that can only spoof referrers cannot fraudulently access hardware, since the browser must also steal another domain's token or retrieve a new one from the hardware device server. However, nothing prevents a browser from autonomously downloading a new authentication in the background under the guise of an authorized domain, and then using this authentication in its HTTP request to the hardware device server.

Although the device protocol translator hides the details of the HTTP protocol used for accessing the hardware device server from the web application, the web application can still issue HTTP requests directly to the hardware device server without going through the device protocol translator. This may disrupt logic in the device protocol translator that buffers device output or otherwise attempts to optimize device accesses. If the browser is not compromised, access attempts can be forced to go through the device protocol translator using techniques by splitting the device protocol translator into two pieces, the public component that exports a standardized interface and a private component that is generated by the hardware device server and handles platform-specific optimizations and quirks. For example, if the device protocol translator is implemented as a JavaScript library running inside a web browser, the library can create an HTML frame in the localhost domain that cannot be accessed by the outer application frame that runs in the domain of the application's originating web server. The private component can be placed in the localhost frame, and the public component can be placed in the enclosing application frame, allowing the two frames to pass messages to each other, but keeping secrets in the localhost frame safe from the untrusted outer frame. The application passes device access requests to the public component; in turn, the public component passes the request to the private component. The private component, which has established a secret with the device server, includes this secret in the forwarded application-generated device request. Thus, the device server can detect that the request originated from the private component, and not from a web application that is trying to directly issue requests without going through the device protocol translator.

Hardware Device Server Implementation

The device protocol translator encodes device requests to the hardware device server as HTTP commands using simple XML strings embedded in HTTP requests. Whereas API calls (147 inFIG. 1) constitute the interface between a web application (146 inFIG. 1) and the device protocol translator (148 inFIG. 1), the HTTP commands (154 and136 inFIG. 1) described herein constitute the interface between the device protocol translator (148 inFIG. 1) and the hardware device server (156 and124 inFIG. 1). The API calls shown in Table 1 and illustrated inFIG. 3 define the interface (see147 inFIG. 1) between a web application and the device protocol translator (see146 and148, respectively, inFIG. 1). The device protocol translator (148 inFIG. 1) transmits commands using HTTP protocol (154 and136 inFIG. 1) to a local hardware device server and remotely connected hardware device server (124 and156 inFIG. 1) for controlling various hardware devices. The commands using HTTP protocol are described below in relation to Table 2. Table 2 is an XML template for accessing the hardware device server.

An exemplary XML string for a command sent to a hardware device server from a device protocol translator requesting an action and a corresponding result sent from the hardware device server to the device protocol translator that includes audio data generated by a microphone are shown in Table 2. Each request includes a command and a corresponding result. A command may include a security authenticate as described above, an action to be performed such as “record”, a target device such as a “microphone”, and optional device-specific params field such as recording “durationLength” and optional data field to be sent to recipient devices such as memory devices. The optional data field is not used in this example. When a hardware device server responds to a command, the responses are also encoded as a simple XML string. In response to the exemplary command shown in Table 2, a result may be returned to the device protocol translator from the hardware device server that may include a value field indicating whether the associated command succeeded or failed. The result may also include a data field such as the requested audio data from a microphone.

	TABLE 2

	<command>

	<authenticate>0fdb53ac3d2...</authenticate>
	<action>record</action>
	<device>microphone</device>
	<params>durationLength=10</params>
	<data></data>

	</command>
	<result>

	<value>OK</value>
	<data>bcd32abfe029e...</data>

	</result>

On the client side, the device protocol translator encapsulates the low level decoding routines and other protocol functionality and responds to API calls from a web page with any requested data or status information.

Although the subject matter has been described in language specific to structural features and methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.