US11210198B2 - Distributed web page performance monitoring methods and systems - Google Patents

Abstract

Computing systems, database systems, and related methods are provided for tracking performance associated with loading or updating a web page in a distributed manner. An exemplary method involves a server providing the web page to a client over a network. The web page includes code executable by the client to provide requests to the server over the network, assign identifiers to the requests, and determine client-side performance metrics associated with the requests. The server provides one or more responses corresponding to the requests to the client over the network, determines server-side performance metrics associated with providing the responses, receives the client-side performance metrics from the client after providing the responses, and maintains the associations between the client-side performance metrics, the server-side performance metrics, and the respective server requests using the identifiers assigned by the client.

Description

TECHNICAL FIELD

Embodiments of the subject matter described herein relate generally to cloud computing and database systems, and more particularly, to methods and systems that support distributed monitoring of web page performance.

BACKGROUND

Modern software development is evolving away from the client-server model toward network-based processing systems that provide access to data and services via the Internet or other networks. In contrast to traditional systems that host networked applications on dedicated server hardware, a “cloud” computing model allows applications to be provided over the network “as a service” or “on-demand” by an infrastructure provider. The infrastructure provider typically abstracts the underlying hardware and other resources used to deliver a customer-developed application so that the customer no longer needs to operate and support dedicated server hardware. The cloud computing model can often provide substantial cost savings to the customer over the life of the application because the customer no longer needs to provide dedicated network infrastructure, electrical and temperature controls, physical security and other logistics in support of dedicated server hardware.

It is desirable to minimize the amount of time or delay involved when a client attempts to load a web page or obtain other data from the cloud. However, monitoring performance of web page loading, data retrieval, and the like utilizes resources, which, in turn, further increases the amount of time or delay perceived by a user, thereby degrading the user experience. Accordingly, it is desirable to monitor performance in a manner that maintains a satisfactory user experience.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete understanding of the subject matter may be derived by referring to the detailed description and claims when considered in conjunction with the following figures, wherein like reference numbers refer to similar elements throughout the figures.

FIG. 1 is a block diagram of an exemplary computing system;

FIG. 2 is a flow diagram of an exemplary client-side monitoring process suitable for implementation in the computing system ofFIG. 1 in accordance with one or more embodiments;

FIG. 3 depicts an exemplary sequence of communications within the computing system ofFIG. 1 in connection with one or more embodiments of the client-side monitoring process ofFIG. 2; and

FIG. 4 is a block diagram of an exemplary multi-tenant system suitable for use with the computing system ofFIG. 1 in accordance with one or more embodiments.

DETAILED DESCRIPTION

Embodiments of the subject matter described herein generally relate to database systems and methods for tracking the performance associated with the retrieval and processing of a computer file, such as a web page, over a communications network. In exemplary embodiments, a server provides a web page that includes code executable by web browser or similar application at a client device to provide one or more requests to the server over the network for data, files, and/or other information required to generate or otherwise present the web page. The code also causes the client application to monitor and determine a set of performance metrics associated with those server requests.

For example, as described in greater detail below in the context ofFIGS. 1-3, in exemplary embodiments, the code includes a script or reference to a computer file that includes scripting language (e.g., a JavaScript file) provided within an initial portion of the web page code that, when executed by the client application, cause the client application to assign identifiers to subsequent server requests and track the amount of time associated with those server requests. In this regard, for each request provided to the server, the client application may determine a browser processing time (BPT) associated with that request based on the amount of time elapsed from when the client application initiates the server request until the client application finishes processing the received server response to that request. In exemplary embodiments, the code also causes the client application to track the amount of time elapsed from when an initial triggering event for loading or updating the web page occurs until the client application finishes loading or processing the web page code, alternatively referred to herein as the experienced page time (EPT). In this regard, the EPT represents a user's perceived amount of time required for the web page to load or update before the user can subsequently interact with the web page.

As described in greater detail below primarily in the context ofFIGS. 1 and 3, in exemplary embodiments, the client application at the client device communicates with an application program interface (API) at the server to support server-side performance tracking. In this regard, for each server request by the client application, the performance monitoring API at the server receives the identifier assigned to that server request and then tracks the amount of time or other performance metrics associated with the server processing and responding to the server request, such as, for example, the elapsed time between initiating processing of the request and transmitting a response back to the client application at the client device, alternatively referred to herein as the server processing time (SPT). The performance monitoring API stores or otherwise maintains the server performance metrics in association with the respective server requests. In this regard, in exemplary embodiments, after the client application finishes loading or updating the web page, the client application transmits or otherwise provides the BPT and other client-side performance metrics associated with the various server requests to the performance monitoring API for storage and/or maintenance in association with the server-side performance metrics. For example, the performance monitoring API may generate a log entry in a database or other data storage element that associates the client-side and server-side performance metrics with the identifier to the corresponding server request, and thereby uniformly formats and maintains the relationships between client-side and server-side performance metrics for retrospective analysis.

By virtue of the performance monitoring being distributed among the client and server sides, the perceived performance degradation by a user on the client-side is reduced by balancing the resources used for performance monitoring relative to overburdening any one of the client or server sides. Additionally, having both client and server sides track performance metrics associated with individual requests provides improved granularity and differentiation between the amount of time attributable to the client-side processing versus the amount of time attributable to the server-side processing. The performance monitoring API is also capable of assembling and formatting the performance data in a manner that allows for the performance data to be integrated or incorporated into any number of different database systems for storage or visualizations. In this regard, maintaining associations between server requests and corresponding performance metrics as well as differentiating between client- and server-side performance metrics allows for drill downs into the details of what request, behavior, or actor is the primary contributor to the overall timing.

FIG. 1 depicts an exemplary embodiment of acomputing system100 capable of monitoring performance associated with aclient device106 retrieving or otherwise accessing a computer file provided by or on aserver102 communicatively coupled to theclient device106 over acommunications network108. Thecommunications network108 may be realized as any sort or combination of wired and/or wireless computer network, a cellular network, a mobile broadband network, a radio network, the Internet, or the like. It should be noted thatFIG. 1 is a simplified representation of acomputing system100 for purposes of explanation and is not intended to be limiting. For example, in practice, multiple instances ofclient devices106 communicating on thenetwork108 may be present, with any number of instances of applications being provided by theserver102 to various instances ofclient devices106. Additionally, practical implementations may include multiple instances of theserver102 and/or thedatabase104, which, in turn may reside behind or rely on one or more load balancers to manage resource utilization, as will be appreciated in the art.

Theserver102 generally represents a computing device, computing system or another combination of processing logic, circuitry, hardware, and/or other components configured to support the processes, tasks, operations, and/or functions described herein. In this regard, theserver102 includes aprocessing system120, which may be implemented using any suitable processing system and/or device, such as, for example, one or more processors, central processing units (CPUs), controllers, microprocessors, microcontrollers, processing cores and/or other hardware computing resources configured to support the operation of theprocessing system120 described herein. Theprocessing system120 may include or otherwise access a data storage element122 (or memory) capable of storing programming instructions for execution by theprocessing system120, that, when read and executed, causeprocessing system120 to support the processes described herein. Depending on the embodiment, thememory122 may be realized as a random-access memory (RAM), read only memory (ROM), flash memory, magnetic or optical mass storage, or any other suitable non-transitory short or long-term data storage or other computer-readable media, and/or any suitable combination thereof. In one or more embodiments, the programming instructions cause theprocessing system120 to create, generate, or otherwise facilitate anapplication platform124 that is capable of supporting, generating, or otherwise providing instances of avirtual application126 at run-time (or “on-demand”) based at least in part upon code and other data that is stored or otherwise maintained by thedatabase104, such that theserver102 functions as anapplication server102. In exemplary embodiments, the programming instructions also cause theprocessing system120 to create, generate, or otherwise facilitate a performance monitoring application program interface (API)128 that supports tracking or otherwise monitoring performance metrics associated with theserver102 providing web pages or other files, data, and/or information to theclient device106, as described in greater detail below. Depending on the embodiment, theperformance monitoring API128 can be integrated with or otherwise incorporated as part of avirtual application126, or be realized as a separate or standalone component that is capable of interacting with theclient device106 independent of anyvirtual application126.

Theclient device106 generally represents an electronic device coupled to thenetwork108 that may be utilized by a user to access theapplication platform124 on theapplication server102 to thereby access instances ofvirtual applications126 supported by theapplication server102 and/or retrieve data from thedatabase104 via thenetwork108. In practice, theclient device106 can be realized as any sort of personal computer, mobile telephone, tablet or other network-enabled electronic device. In exemplary embodiments, theclient device106 includes a display device, such as a monitor, screen, or another conventional electronic display, capable of graphically presenting data and/or information along with a user input device, such as a touchscreen, a touch panel, a mouse, a joystick, a directional pad, a motion sensor, or the like, capable of receiving input from the user of theclient device106. The illustratedclient device106 executes or otherwise supports aclient application107 that communicates with theapplication platform124 on theserver102 using a networking protocol. In some embodiments, theclient application107 is realized as a web browser or similar local client application executed by theclient device106 that contacts theapplication server102 and/orapplication platform124 using a networking protocol, such as the hypertext transport protocol (HTTP) or the like.

In one or more embodiments, a user utilizes abrowser application107 to access or otherwise retrieve a computer file, such as a HyperText Markup Language (HTML) file corresponding to a web page that is generated by theapplication server102 and/or theapplication platform124 as part of an instance of avirtual application126 generated at run-time (or “on-demand”) based at least in part on data stored or otherwise maintained by thedatabase104 that is communicatively coupled to theserver102 via thecommunications network108. It should be noted that although one or more embodiments may be described herein in the context of a web page integrated in a virtual application in an on-demand database system, the subject matter is not necessarily so limited, and in practice, may be implemented in an equivalent manner in any number of different database systems, cloud computing systems, and the like, and in the context of any type of document, computer file or other resource provided by a server that includes, presents, or otherwise displays elements at a client device over a network.

In exemplary embodiments, thedatabase104 stores or otherwise maintains data for integration with or invocation by a virtual application in objects organized in object tables. In this regard, thedatabase104 includes a plurality of different object tables configured to store or otherwise maintain alphanumeric values, metadata, or other descriptive information that define a particular instance of a respective type of object associated with a respective object table. For example, the virtual application may support a number of different types of objects that may be incorporated into or otherwise depicted or manipulated by the virtual application, with each different type of object having a corresponding object table that includes columns or fields corresponding to the different parameters or criteria that define a particular instance of that object. In one embodiment, thedatabase104 stores or otherwise maintains application objects (e.g., an application object type) where the application object table includes columns or fields corresponding to the different parameters or criteria that define aparticular application126 capable of being generated or otherwise provided by theapplication platform124 on aclient device106. In this regard, thedatabase104 may also store or maintain graphical user interface (GUI) objects that may be associated with or referenced by a particular application object and include columns or fields that define the layout, sequencing, and other characteristics of GUI displays to be presented by theapplication platform124 on aclient device106 in conjunction with thatapplication126. Additionally, thedatabase104 stores or otherwise maintains additional database objects for association and/or integration with theapplication126. Thedatabase104 also stores or otherwise maintains metadata, which may be utilized to perform data manipulation and/or formatting. For example, the metadata may include or define describe any number of workflows, process flows, formulas, business logic, structure and other database components or constructs that may be associated with a particular application database object.

In one or more embodiments, thedatabase104 stores or otherwise maintains a monitoring script file130 (e.g., a JavaScript file or similar file including scripting language) that may be incorporated by reference into the HTML file corresponding to a web page to be generated by theapplication server102 and/or theapplication platform124 to facilitate tracking the performance associated with that web page. For example, a developer may provide a reference to themonitoring script file130 within the HTML code for a web page to be presented in connection with thevirtual application126 to facilitate tracking of performance of that web page at run-time during generation or presentation of the web page by theclient application107 at theclient device106. In this regard, theclient application107 may include an engine or interpreter configured to execute the script or code provided by themonitoring script file130 maintained in thedatabase104, and thereby initiate the performance monitoring processes described herein. In this manner, developers may enable performance tracking of a particular web page in a drop-in manner by adding reference to themonitoring script file130 maintained by thedatabase104. Such embodiments also allow for the performance monitoring processes and correspondingperformance monitoring API128 to be implemented and maintained independent of the web page code.

FIG. 2 depicts an exemplary embodiment of a client-side monitoring process200 suitable for implementation in thecomputing system100 to track or otherwise monitor performance of a web page at aclient device106. For illustrative purposes, the following description may refer to elements mentioned above in connection withFIG. 1. In this regard, while portions of the client-side monitoring process200 may be performed by different elements of thecomputing system100, for purposes of explanation, the subject matter is described herein in the context of the client-side monitoring process200 being primarily performed by theclient device106 and/or theclient application107. It should be appreciated that the client-side monitoring process200 may include any number of additional or alternative tasks, the tasks need not be performed in the illustrated order and/or the tasks may be performed concurrently, and/or the client-side monitoring process200 may be incorporated into a more comprehensive procedure or process having additional functionality not described in detail herein. Moreover, one or more of the tasks shown and described in the context ofFIG. 2 could be omitted from a practical embodiment of the client-side monitoring process200 as long as the intended overall functionality remains intact.

In the illustrated embodiment, the client-side monitoring process200 monitors for a triggering event for tracking the performance of a web page (task202). In this regard, the triggering event could be an initial page loading, a refresh of the web page or other dynamic update to the web page, or another triggering event associated with the web page that may be detected or otherwise identified by theclient application107. For example, the monitoring script may cause theclient application107 to monitor for a mouse click event or other user input events that are configured to initiate the performance monitoring described herein. In this regard, the monitoring script may specify browser events that trigger the performance monitoring processes, such as, for example, mouse click events, keyboard events, JavaScript animations, and the like. In response to detecting a triggering event, the client-side monitoring process200 identifies or otherwise obtains an initial time associated with the client-side processing responsive to the triggering event (task204). For example, theclient application107 may store or otherwise maintain the timestamp associated with a mouse click event, a web page refresh or the like as the initial reference time for determining the experienced page time (EPT) or other performance metrics characterizing the overall user experience or time required to finish responding to the triggering event.

The client-side monitoring process200 continues by generating or otherwise creating unique identifiers to be assigned to different requests provided to the server in connection with responding to the triggering event and then tracks, obtains, or otherwise records browser performance times associated with those requests using the assigned identifiers (tasks206,208). In this regard, the monitoring script executed by theclient application107 is configured to identify XMLHttpRequest (XHR) calls or other requests for information from theserver102 within the web page code during execution, and for each request made to theserver102, assign a unique tracing identifier that may be utilized to track performance times associated with that server request. For example, theclient application107 may store or otherwise record the timestamp corresponding to the time when theclient application107 transmits the server request via thenetwork108, the timestamp corresponding to the time when theclient application107 receives a response to the request from theserver102 via thenetwork108, and the timestamp corresponding to the time when theclient application107 finished processing the response (e.g., the time when rendering within the web browser GUI display is finished) in association with the tracing identifier for that request. Additionally, in exemplary embodiments, theclient application107 also transmits or otherwise provides the tracing identifier for each server request to theperformance monitoring API128 in connection with providing the request to theserver102. As described in greater detail below, theperformance monitoring API128 utilizes the same tracing identifier to monitor or track server-side performance metrics associated with responding to the request so that the client-side and server-side performance metrics for that same request can be subsequently collated for analyzing and/or drilling down into performance associated with that request.

In exemplary embodiments, the repeats the tasks of assigning tracing identifiers to server requests and tracking client-side performance metrics associated with those server requests until all of the server requests associated loading or updating the web page in response to the triggering event have been completed (task210). Thereafter, the client-side monitoring process200 detects or otherwise identifies when the loading of the web page is fully complete prior to determining the overall performance metrics associated with the web page (task212). In this regard, the monitoring script waits for any rendering or other thrashing by theclient application107 is completed before identifying the resulting timestamp as the time when the processing of the web page is finished. Based on the difference relative to the initial timestamp for initiating the loading or updating of the web page, theclient application107 calculates or otherwise determines the experienced page time (EPT) and/or other performance metrics that characterize the overall performance of loading and/or updating the web page.

After the collection or determination of the client-side performance metrics is completed, the client-side monitoring process200 transmits or otherwise provides the set of client-side performance metrics to the server (task214). In this regard, the monitoring script executed by theclient application107 causes theclient application107 to transmit or otherwise provide the performance metrics associated with each of the individual server requests along with the overall performance metrics to theperformance monitoring API128 at theserver102 for collating and maintaining the client-side performance metrics in association with server-side performance metrics, as described in greater detail below.

FIG. 3 depicts anexemplary sequence300 of communications within thecomputing system100 in accordance with one or more embodiments. As described above, the illustratedsequence300 begins with theclient application107 at theclient device106 downloading, accessing, or otherwise obtaining302 amonitoring script file130 maintained in thedatabase104 that is referred to or otherwise incorporated into the code for the web page associated with thevirtual application126 to be provided at theclient device106. For example, a user may input or otherwise provide a network address associated with a web page file for thevirtual application126 provided by theserver102 into thebrowser application107, which, in turn results in thebrowser application107 contacting theserver102 over thenetwork108 to obtain the web page file associated with thevirtual application126. In one or more embodiments, the web page code includes a reference to themonitoring script file130 in thedatabase104 or otherwise incorporates the executable script from themonitoring script file130 within the web page code that precedes subsequent executable code portions for requesting data from theserver102 and/ordatabase104 for rendering the web page. Thus, during execution and/or rendering of the web page code by thebrowser application107, thebrowser application107 may request themonitoring script file130 from thedatabase104 via theserver102 and/orvirtual application126 prior to executing subsequent code for rendering the web page. Execution of themonitoring script130 by theclient application107 during execution of the web page code causes theclient application107 to monitor for an event that triggers loading, updating, or other changes to the web page presented by theclient application107, and after detecting a triggering event, assign unique tracing identifiers to server requests during subsequent execution of the web page code. In this regard, in response to a triggering event, theclient application107 may record or otherwise store the timestamp associated with the triggering event before generating a unique tracing identifier to be assigned to an initial server request. Theclient application107 may also record or otherwise store the timestamp associated with transmitting304 the request including the unique tracing identifier to theapplication server102 and/orapplication platform124.

Themonitoring API128 receives the request and utilizes the unique tracing identifier provided by theclient application107 to manage the tracking and determination of server-side performance metrics for responding to the request. For example, when the server request involves a call to the database104 (e.g., to obtain data or information for populating a portion of the GUI display associated with the web page), themonitoring API128 may transmit or otherwise provide306 the unique tracing identifier along with request to thevirtual application126, which, in turn, queries or otherwise accesses310 thedatabase104 to obtain the data for responding to the request from the client, and then processes or otherwise acts on the data obtained from thedatabase104 before transmitting or otherwise providing312 a response to the client that includes or is otherwise influenced by the data obtained from thedatabase104. While thevirtual application126 is in the process of responding to the request, thevirtual application126 and/or themonitoring API128 records or stores, in association with the unique tracing identifier assigned to the request, the timestamps associated with one or more of the following: thevirtual application126 receiving the client's request, thevirtual application126 initiating the call to thedatabase104, thevirtual application126 receiving data from thedatabase104, and thevirtual application126 transmitting the server response to the client request. Other metrics associated with a call to thedatabase104 may also be recorded, such as, for example, a number of rows created, read, updated, deleted, and/or the like. In some embodiments, thedatabase104 may also track database performance by receiving the unique tracing identifier from themonitoring API128 and/or thevirtual application126 and recording or storing, in association with the unique tracing identifier assigned to the request from theserver102, the timestamp associated with the initial receipt of the query or call from thevirtual application126 along with the timestamp associated with thedatabase104 transmitting the responsive data back to thevirtual application126, the number of rows or tables accessed or impacted by the request, and/or the like.

It should be noted that in addition to server-side tracking of database performance metrics, thevirtual application126 may similarly track performance metrics or timings associated with calls or requests sent to other servers, systems or devices on thenetwork108. Additionally, in some embodiments, such other external servers or systems could support themonitoring API128 and/or the monitoring processes described herein, such that they independently track performance metrics using tracing identifiers provided by themonitoring API128 and/or thevirtual application126 in connection with the request, and providing the performance metrics back to themonitoring API128 and/or thevirtual application126 after or in conjunction with responding to the request.

After receiving the server response, theclient application107 stores or otherwise records the timestamp associated with receipt of the server response in association with the tracing identifier and then continues processing or otherwise acting on the received server response to continue loading or updating the web page at theclient device106. After all of the server requests are completed and the web page has finished loading or updating, theclient application107 transmits or otherwise provides314 the various client-side performance metrics or timestamps associated with the unique tracing identifier to themonitoring API128 at theserver102. In the illustrated embodiment, themonitoring API128 also receives or otherwise obtains316 the various server-side performance metrics or timestamps associated with the unique tracing identifier from thevirtual application126. In embodiments where thedatabase104 independently tracks performance metrics at thedatabase104, themonitoring API128 may also receive or otherwise obtain the various database performance metrics or timestamps associated with the unique tracing identifier from thedatabase104. Themonitoring API128 then collates or otherwise converts the various timestamps or performance metrics into a universally formatted object or message. In the illustrated embodiment, themonitoring API128 creates an entry in a performance log in thedatabase104 that is associated with the web page (or the network address thereof) and also stores or otherwise maintains the various client-side, server-side, and/or database performance metrics associated with particular server requests in association with the unique tracing identifiers assigned to the respective requests. For example, in one embodiment, each instance of a web page loading in a browser tab is assigned an initial identifier upon initiation of the client-side monitoring process200, which is then utilized as a parent identifier for associating the various requests and tracing identifiers that thereafter emanate from that web page browser session in a single log entry associated with that session. In this regard, it should be noted that any number of different hierarchical or parent-child relationships may be created that enable establishing temporal or logical associations between requests as well as tracking performance of a particular web page with respect to time.

In various embodiments, themonitoring API128 may transmit or otherwise provide the performance metrics to another external system in a universally formatted for ingestion, integration or other processing by that system. For example, the monitoring API may generate a message formatted using a JavaScript Object Notation (JSON) data format that includes the client-side, server-side and database performance metrics and the unique tracing identifiers, with the JSON data message being parsed, processed, ingested or otherwise integrated by another system on thenetwork108.

By virtue of the timings or performance metrics being independently tracked at each of theclient device106 and theserver102, the load associated with performance monitoring at any one of thecomponents102,106 in thecomputing system100 may be minimized, thereby allowing a developer to incorporate performance monitoring without significantly degrading the user experience. Moreover, the different performance metrics or timestamps may be utilized to granularly drill down into the performance of a particular request to identify which of thecomponents102,104,106 required a disproportionate or undesirably long amount of time during execution of the request, or conversely, to identify which of thecomponents102,104,106 is most expeditiously executing its workload. In this regard, some embodiments may allow a developer to isolate performance metrics to individual users, web pages, database calls, or the like, and/or particular combinations thereof. Thus, a developer may more efficiently identify what issues may need to be addressed and proactively respond (e.g., rather than waiting for complaints by end users or customers). The timings or performance metrics may also be analyzed with respect to time for a common server request, thereby allowing a developer to track performance with respect to time or other benchmarks (e.g., service level agreements).

It should be noted that the subject matter described herein is not limited to any particular number, type, or combination of performance metrics being tracked. That said, in one or more exemplary embodiments, the client-side performance metrics include the time between an initial fetch request until a static HTML was received, a time required to initially render a web page (bootstrap time), a timestamp for the start of an XHR call, a time until receiving a server reply to the XHR call, and a timestamp for when rendering responsive to the XHR call is completed. Exemplary server-side performance metrics include the time from when the server starts processing a request until when its response is finished being sent to the client.

FIG. 4 depicts an exemplary embodiment of an on-demandmulti-tenant database system400 suitable for use with thecomputing system100 ofFIG. 1 and the monitoring processes described herein. The illustratedmulti-tenant system400 ofFIG. 4 includes aserver402, such asserver102, that dynamically creates and supportsvirtual applications428 based upondata432 from a common database430 (e.g., database104) that is shared between multiple tenants, alternatively referred to herein as a multi-tenant database. Data and services generated by thevirtual applications428 are provided via a network445 (e.g., network108) to any number of client devices440 (e.g., client device106), as desired. Eachvirtual application428 is suitably generated at run-time (or on-demand) using a common application platform410 (e.g., application platform124) that securely provides access to thedata432 in thedatabase430 for each of the various tenants subscribing to themulti-tenant system400. In this regard, depending on the embodiment, the client-side monitoring process200 may be utilized in connection withvirtual applications428 provided by or on theapplication platform410, or avirtual application428 generated by theapplication platform410 may be utilized to review and analyze performance data maintained in thedatabase430. In accordance with one non-limiting example, themulti-tenant system400 is implemented in the form of an on-demand multi-tenant customer relationship management (CRM) system that can support any number of authenticated users of multiple tenants.

As used herein, a “tenant” or an “organization” should be understood as referring to a group of one or more users that shares access to common subset of the data within themulti-tenant database430. In this regard, each tenant includes one or more users associated with, assigned to, or otherwise belonging to that respective tenant. To put it another way, each respective user within themulti-tenant system400 is associated with, assigned to, or otherwise belongs to a particular tenant of the plurality of tenants supported by themulti-tenant system400. Tenants may represent customers, customer departments, business or legal organizations, and/or any other entities that maintain data for particular sets of users within the multi-tenant system400 (i.e., in the multi-tenant database430). For example, theapplication server402 may be associated with one or more tenants supported by themulti-tenant system400. Although multiple tenants may share access to theserver402 and thedatabase430, the particular data and services provided from theserver402 to each tenant can be securely isolated from those provided to other tenants (e.g., by restricting other tenants from accessing a particular tenant's data using that tenant's unique organization identifier as a filtering criterion). The multi-tenant architecture therefore allows different sets of users to share functionality and hardware resources without necessarily sharing any of thedata432 belonging to or otherwise associated with other tenants.

Themulti-tenant database430 is any sort of repository or other data storage system capable of storing and managing thedata432 associated with any number of tenants. Thedatabase430 may be implemented using any type of conventional database server hardware. In various embodiments, thedatabase430shares processing hardware404 with theserver402. In other embodiments, thedatabase430 is implemented using separate physical and/or virtual database server hardware that communicates with theserver402 to perform the various functions described herein. In an exemplary embodiment, thedatabase430 includes a database management system or other equivalent software capable of determining an optimal query plan for retrieving and providing a particular subset of thedata432 to an instance ofvirtual application428 in response to a query initiated or otherwise provided by avirtual application428. Themulti-tenant database430 may alternatively be referred to herein as an on-demand database, in that themulti-tenant database430 provides (or is available to provide) data at run-time to on-demandvirtual applications428 generated by theapplication platform410.

In practice, thedata432 may be organized and formatted in any manner to support theapplication platform410. In various embodiments, thedata432 is suitably organized into a relatively small number of large data tables to maintain a semi-amorphous “heap”-type format. Thedata432 can then be organized as needed for a particularvirtual application428. In various embodiments, conventional data relationships are established using any number of pivot tables434 that establish indexing, uniqueness, relationships between entities, and/or other aspects of conventional database organization as desired. Further data manipulation and report formatting is generally performed at run-time using a variety of metadata constructs. Metadata within a universal data directory (UDD)436, for example, can be used to describe any number of forms, reports, workflows, user access privileges, business logic and other constructs that are common to multiple tenants. Tenant-specific formatting, functions and other constructs may be maintained as tenant-specific metadata438 for each tenant, as desired. Rather than forcing thedata432 into an inflexible global structure that is common to all tenants and applications, thedatabase430 is organized to be relatively amorphous, with the pivot tables434 and themetadata438 providing additional structure on an as-needed basis. To that end, theapplication platform410 suitably uses the pivot tables434 and/or themetadata438 to generate “virtual” components of thevirtual applications428 to logically obtain, process, and present the relativelyamorphous data432 from thedatabase430.

Still referring toFIG. 4, theserver402 is implemented using one or more actual and/or virtual computing systems that collectively provide thedynamic application platform410 for generating thevirtual applications428. For example, theserver402 may be implemented using a cluster of actual and/or virtual servers operating in conjunction with each other, typically in association with conventional network communications, cluster management, load balancing and other features as appropriate. Theserver402 operates with any sort ofconventional processing hardware404, such as aprocessor405,memory406, input/output features407 and the like. The input/output features407 generally represent the interface(s) to networks (e.g., to thenetwork445, or any other local area, wide area or other network), mass storage, display devices, data entry devices and/or the like. Theprocessor405 may be implemented using any suitable processing system, such as one or more processors, controllers, microprocessors, microcontrollers, processing cores and/or other computing resources spread across any number of distributed or integrated systems, including any number of “cloud-based” or other virtual systems. Thememory406 represents any non-transitory short or long-term storage or other computer-readable media capable of storing programming instructions for execution on theprocessor405, including any sort of random-access memory (RAM), read only memory (ROM), flash memory, magnetic or optical mass storage, and/or the like. The computer-executable programming instructions, when read and executed by theserver402 and/orprocessor405, cause theserver402 and/orprocessor405 to create, generate, or otherwise facilitate theapplication platform410 and/orvirtual applications428 and perform one or more additional tasks, operations, functions, and/or processes described herein. It should be noted that thememory406 represents one suitable implementation of such computer-readable media, and alternatively or additionally, theserver402 could receive and cooperate with external computer-readable media that is realized as a portable or mobile component or application platform, e.g., a portable hard drive, a USB flash drive, an optical disc, or the like.

Theapplication platform410 is any sort of software application or other data processing engine that generates thevirtual applications428 that provide data and/or services to theclient devices440. In a typical embodiment, theapplication platform410 gains access to processing resources, communications interfaces and other features of theprocessing hardware404 using any sort of conventional orproprietary operating system408. Thevirtual applications428 are typically generated at run-time in response to input received from theclient devices440. For the illustrated embodiment, theapplication platform410 includes a bulkdata processing engine412, aquery generator414, asearch engine416 that provides text indexing and other search functionality, and aruntime application generator420. Each of these features may be implemented as a separate process or other module, and many equivalent embodiments could include different and/or additional features, components or other modules as desired.

Theruntime application generator420 dynamically builds and executes thevirtual applications428 in response to specific requests received from theclient devices440. Thevirtual applications428 are typically constructed in accordance with the tenant-specific metadata438, which describes the particular tables, reports, interfaces and/or other features of theparticular application428. In various embodiments, eachvirtual application428 generates dynamic web content that can be served to a browser orother client program442 associated with itsclient device440, as appropriate.

Theruntime application generator420 suitably interacts with thequery generator414 to efficiently obtainmulti-tenant data432 from thedatabase430 as needed in response to input queries initiated or otherwise provided by users of theclient devices440. In a typical embodiment, thequery generator414 considers the identity of the user requesting a particular function (along with the user's associated tenant), and then builds and executes queries to thedatabase430 using system-wide metadata436, tenantspecific metadata438, pivot tables434, and/or any other available resources. Thequery generator414 in this example therefore maintains security of thecommon database430 by ensuring that queries are consistent with access privileges granted to the user and/or tenant that initiated the request. In this manner, thequery generator414 suitably obtains requested subsets ofdata432 accessible to a user and/or tenant from thedatabase430 as needed to populate the tables, reports or other features of the particularvirtual application428 for that user and/or tenant.

Still referring toFIG. 4, thedata processing engine412 performs bulk processing operations on thedata432 such as uploads or downloads, updates, online transaction processing, and/or the like. In many embodiments, less urgent bulk processing of thedata432 can be scheduled to occur as processing resources become available, thereby giving priority to more urgent data processing by thequery generator414, thesearch engine416, thevirtual applications428, etc.

In exemplary embodiments, theapplication platform410 is utilized to create and/or generate data-drivenvirtual applications428 for the tenants that they support. Suchvirtual applications428 may make use of interface features such as custom (or tenant-specific)screens424, standard (or universal) screens422 or the like. Any number of custom and/orstandard objects426 may also be available for integration into tenant-developedvirtual applications428. As used herein, “custom” should be understood as meaning that a respective object or application is tenant-specific (e.g., only available to users associated with a particular tenant in the multi-tenant system) or user-specific (e.g., only available to a particular subset of users within the multi-tenant system), whereas “standard” or “universal” applications or objects are available across multiple tenants in the multi-tenant system. For example, a virtual CRM application may utilizestandard objects426 such as “account” objects, “opportunity” objects, “contact” objects, or the like. Thedata432 associated with eachvirtual application428 is provided to thedatabase430, as appropriate, and stored until it is requested or is otherwise needed, along with themetadata438 that describes the particular features (e.g., reports, tables, functions, objects, fields, formulas, code, etc.) of that particularvirtual application428. For example, avirtual application428 may include a number ofobjects426 accessible to a tenant, wherein for eachobject426 accessible to the tenant, information pertaining to its object type along with values for various fields associated with that respective object type are maintained asmetadata438 in thedatabase430. In this regard, the object type defines the structure (e.g., the formatting, functions and other constructs) of eachrespective object426 and the various fields associated therewith.

Still referring toFIG. 4, the data and services provided by theserver402 can be retrieved using any sort of personal computer, mobile telephone, tablet or other network-enabledclient device440 on thenetwork445. In an exemplary embodiment, theclient device440 includes a display device, such as a monitor, screen, or another conventional electronic display capable of graphically presenting data and/or information retrieved from themulti-tenant database430. Typically, the user operates a conventional browser application or other client program442 (e.g., client application107) executed by theclient device440 to contact theserver402 via thenetwork445 using a networking protocol, such as the hypertext transport protocol (HTTP) or the like. The user typically authenticates his or her identity to theserver402 to obtain a session identifier (“SessionID”) that identifies the user in subsequent communications with theserver402. When the identified user requests access to avirtual application428, theruntime application generator420 suitably creates the application at run time based upon themetadata438, as appropriate. As noted above, thevirtual application428 may contain Java, ActiveX, or other content that can be presented using conventional client software running on theclient device440; other embodiments may simply provide dynamic web or other content that can be presented and viewed by the user, as desired.

The foregoing description is merely illustrative in nature and is not intended to limit the embodiments of the subject matter or the application and uses of such embodiments. Furthermore, there is no intention to be bound by any expressed or implied theory presented in the technical field, background, or the detailed description. As used herein, the word “exemplary” means “serving as an example, instance, or illustration.” Any implementation described herein as exemplary is not necessarily to be construed as preferred or advantageous over other implementations, and the exemplary embodiments described herein are not intended to limit the scope or applicability of the subject matter in any way.

For the sake of brevity, conventional techniques related to web pages and/or browsers, APIs, software development, testing, performance monitoring, tracing, multi-tenancy, cloud computing, on-demand applications, and other functional aspects of the systems (and the individual operating components of the systems) may not be described in detail herein. In addition, those skilled in the art will appreciate that embodiments may be practiced in conjunction with any number of system and/or network architectures, data transmission protocols, and device configurations, and that the system described herein is merely one suitable example. Furthermore, certain terminology may be used herein for the purpose of reference only, and thus is not intended to be limiting. For example, the terms “first,” “second” and other such numerical terms do not imply a sequence or order unless clearly indicated by the context.

Embodiments of the subject matter may be described herein in terms of functional and/or logical block components, and with reference to symbolic representations of operations, processing tasks, and functions that may be performed by various computing components or devices. Such operations, tasks, and functions are sometimes referred to as being computer-executed, computerized, software-implemented, or computer-implemented. In practice, one or more processing systems or devices can carry out the described operations, tasks, and functions by manipulating electrical signals representing data bits at accessible memory locations, as well as other processing of signals. The memory locations where data bits are maintained are physical locations that have particular electrical, magnetic, optical, or organic properties corresponding to the data bits. It should be appreciated that the various block components shown in the figures may be realized by any number of hardware, software, and/or firmware components configured to perform the specified functions. For example, an embodiment of a system or a component may employ various integrated circuit components, e.g., memory elements, digital signal processing elements, logic elements, look-up tables, or the like, which may carry out a variety of functions under the control of one or more microprocessors or other control devices. When implemented in software or firmware, various elements of the systems described herein are essentially the code segments or instructions that perform the various tasks. The program or code segments can be stored in a processor-readable medium or transmitted by a computer data signal embodied in a carrier wave over a transmission medium or communication path. The “processor-readable medium” or “machine-readable medium” may include any non-transitory medium that can store or transfer information. Examples of the processor-readable medium include an electronic circuit, a semiconductor memory device, a ROM, a flash memory, an erasable ROM (EROM), a floppy diskette, a CD-ROM, an optical disk, a hard disk, a fiber optic medium, a radio frequency (RF) link, or the like. The computer data signal may include any signal that can propagate over a transmission medium such as electronic network channels, optical fibers, air, electromagnetic paths, or RF links. The code segments may be downloaded via computer networks such as the Internet, an intranet, a LAN, or the like. In this regard, the subject matter described herein can be implemented in the context of any computer-implemented system and/or in connection with two or more separate and distinct computer-implemented systems that cooperate and communicate with one another. In one or more exemplary embodiments, the subject matter described herein is implemented in conjunction with a virtual customer relationship management (CRM) application in a multi-tenant environment.

While at least one exemplary embodiment has been presented in the foregoing detailed description, it should be appreciated that a vast number of variations exist. It should also be appreciated that the exemplary embodiment or embodiments described herein are not intended to limit the scope, applicability, or configuration of the claimed subject matter in any way. Rather, the foregoing detailed description will provide those skilled in the art with a convenient road map for implementing the described embodiment or embodiments. It should be understood that various changes can be made in the function and arrangement of elements without departing from the scope defined by the claims, which includes known equivalents and foreseeable equivalents at the time of filing this patent application. Accordingly, details of the exemplary embodiments or other limitations described above should not be read into the claims absent a clear intention to the contrary.

Claims (20)

What is claimed is:

1. A method of tracking performance associated with a web page, the method comprising:

providing, by a server, the web page to a client over a network, the web page including code executable by the client to provide one or more requests to the server over the network, assign tracing identifiers to the one or more requests and provide the tracing identifiers to the server and second code executable by the client to determine, at the client, a first set of performance metrics associated with the one or more requests based on one or more timestamps corresponding to the one or more requests;

providing, by the server, one or more responses to the one or more requests to the client over the network;

determining, by the server, a second set of performance metrics associated with providing the one or more responses;

receiving, by the server, the first set of performance metrics from the client after providing the one or more responses; and

maintaining, by the server, an association between the first set of performance metrics determined at the client and the second set of performance metrics using the tracing identifiers.

2. The method ofclaim 1, further comprising establishing, by the server, the association between the first set of performance metrics and the second set of performance metrics for a first request of the one or more requests using a first tracing identifier of the tracing identifiers, wherein the code causes the client to assign the first tracing identifier to the first request and transmit the first tracing identifier to the server with the first request.

3. The method ofclaim 2, wherein maintaining the association comprises the server generating a log entry in a database, wherein the log entry maintains the association between the first set of performance metrics, the second set of performance metrics and the first tracing identifier.

4. The method ofclaim 2, wherein maintaining the association comprises the server generating a message including the first set of performance metrics, the second set of performance metrics and the first tracing identifier.

5. The method ofclaim 1, wherein:

determining the second set of performance metrics comprises determining a server-side performance metric based on a timing associated with a first request of the one or more requests;

receiving the first set of performance metrics comprises receiving a client-side performance metric associated with the first request; and

maintaining the association comprises maintaining the association between the server-side performance metric and the client-side performance metric using a tracing identifier associated with the first request.

6. The method ofclaim 5, further comprising:

instructing, by the server, a database to determine a database performance metric associated with the first request; and

receiving, by the server, the database performance metric from the database, wherein maintaining the association comprises maintaining the association between the server-side performance metric, the client-side performance metric, and the database performance metric using the tracing identifier.

7. The method ofclaim 1, the code comprising a reference to a script file maintained in a database, wherein providing the web page comprises:

receiving, by the server from the client, a request for the script file during loading of the web page;

obtaining, by the server, the script file from the database; and

providing, by the server, the script file to the client.

8. The method ofclaim 7, wherein the client executes the script file to assign the tracing identifiers to the one or more requests and provide the tracing identifiers to a monitoring application program interface at the server, wherein the monitoring application program interface maintains the association between the first set of performance metrics and the second set of performance metrics using the tracing identifiers.

9. The method ofclaim 7, wherein the script file comprises a JavaScript file.

10. The method ofclaim 7, wherein the reference precedes executable portions of the code for the web page corresponding to the one or more requests.

11. A computer-readable medium having instructions stored thereon that are executable by a processing system of a server to:

provide a web page to a client over a network, the web page including code executable by the client to provide one or more requests to the server over the network, assign tracing identifiers to the one or more requests and provide the tracing identifiers to the server and second code executable by the client to determine, at the client, a first set of performance metrics associated with the one or more requests based on one or more timestamps corresponding to the one or more requests;

provide one or more responses to the one or more requests to the client over the network;

determine a second set of performance metrics associated with providing the one or more responses;

receive the first set of performance metrics from the client after providing the one or more responses; and

maintain an association between the first set of performance metrics determined at the client and the second set of performance metrics using the tracing identifiers.

12. The computer-readable medium ofclaim 11, wherein the code comprises a reference to a script file maintained in a database.

13. The computer-readable medium ofclaim 12, wherein the script file comprises a JavaScript file.

14. The computer-readable medium ofclaim 12, wherein the reference precedes executable portions of the code for the web page corresponding to the one or more requests.

15. A database system comprising:

a database including code corresponding to a web page; and

a server coupled to the database and a network to:

provide the code corresponding to the web page to a client coupled to the network, the code including first code executable by the client to provide one or more requests to the server over the network and assign, to each of the one or more requests, a tracing identifier provided to the server with each of the one or more requests and second code executable by the client to determine, at the client, a first set of performance metrics associated with the one or more requests;

determine a second set of performance metrics associated with responding to the one or more requests;

receive the first set of performance metrics from the client after responding to the one or more requests; and

create a log entry in the database to maintain an association between the tracing identifier for a respective request of the one or more requests, the first set of performance metrics determined at the client for the respective request, and the second set of performance metrics for the respective request.

16. The database system ofclaim 15, wherein the code comprises a script executable by the client.

17. The database system ofclaim 16, wherein the script comprises a file maintained in the database and referenced by the code.

18. The database system ofclaim 17, wherein the file comprises a JavaScript file.

19. The database system ofclaim 15, wherein:

the first set of performance metrics comprises a browser performance time for each of the one or more requests;

the second set of performance metrics comprises a server performance time for each of the one or more requests; and

the log entry maintains the association between [[an]] the tracing identifier assigned to a respective request of the one or more requests, the browser performance time for the respective request, and the server performance time for the respective request.

20. The database system ofclaim 15, wherein a monitoring application program interface at the server receives the first set of performance metrics and creates the log entry.

Images