WO 92/08199 1 PCr/lJS91/08077 f~ ) ,' , J .~J
COMPUTER DOCUMENTS AS COMPOUND
DOCUMENTS IN A NOTEBOOK METAPHOR
Technical Field The present invention is directed towards the problem of constructing, organizing, accessing and linking documents on a computer. Such documents may contain text, graphic images, geometric figures, or numerical spreadsheets, or may be compound documents made up of nested or embedded sub-documents.
Background Art There are many inventions addressing this area in the prior art. Most are based on the idea of a "Graphical User Interface" which attempts to model - some part of the real world on a computer screen. Previous work commonly uses a "desk-top metaphor" in which the computer's display is ~eated as a desk-top upon which objects such as documents or computer programs are represented as symbols or icons. These icons may be "dragged" to any location on the desk-top by using a pointing device such as a mouse. Desk-top objects may be aggregated by collection into any number of folders. These 2 0 folders may be placed inside of other folders. The Xerox Star computer and the Macintosh from Apple Computer are examples of such work. Much of this prior work uses an "application program & data file" view of the world in which the user is aware that an application program is executing which can operate on a number of data files or documents.
2 5 Information in the area of software related to this invention can be found in Inside Macintosh. Fundamentals of Interactive Computer Graphics, SmallTalk-80. The Language and Its Implementation ~all from Addison-Wesley Publishing Co.), "The X Window System" (ACM Transactions on Graphics, Vol. 5, No. 2, April 1986, pages 79-109) or "Andrew: A Distributed Personal Computing Environment" (Communications of the ACM, March 1986). The actual techniques use to recognize both gestures and handwritten input will not be discussed in detail here, but may be found in "Automatic ~. ~ - ' , , .
- : , , :
WO 92/08199 , 2 pc~r/us9l/o8o7?
Recognition of Handprinted Characters - The State of the Art", Proceedings of the IEEE pages 469 187, Vol. 68, No. 4, April 1980. The foregoing references are incorporated herein by reference.
As organizing frameworks, desk-tops and folders leave much to be desired. Objects within these "containers" can be arranged in any pattern with no structuring supported within the container, except further containment or nesting. Indeed, many previous systems support comrnands to "Clean up the window" or "Arrange icons". Often objects may be displayed as a list of titles sorted by name, size, date, or type but the user has no way to impose his arbitrary order on the list.
The "program & file" model intrudes in a number of ways. First: such a design requires a load/save metaphor, in which the user must tell the program to load or save a data file. Second: when one program is operating upon many files, the user must select which file the program's attention 1 5 should be focused on. This is because it must be clear what is the target of shared controls such as command menus.
A number of different approaches to the construction of compound documents exist in prior work. One approach is to simply merge a static image of the foreign document into the parent file. Such an irnage is "dead", 2 0 in that it can no longer be manipulated in its native format. Thus, a text document could contain an embedded pie chart, but the ability to vary the underlying quantities that produced the chart would be lost. These static images are commonly stored as "bit-maps", or arrays of pixels. Bit-maps are difficult to re-size because the underlying structure has been lost.
2 5 Another scheme is to have the parent document's controlling program recognize the imported file's format. For full generality, this requires that all application programs understand all data file formats.
A third approach is to simply provide a window in the parent document in which the program controlling the su~document displays its 3 0 output. In order for both the parent and child documents to be responsive to manipulation, both application programs must be executing. This implies some form of multi-tasking operating system. Such an approach is taken by ...
- . . . . .
-WO 9~/08199 3 PCI/US91/08077 . . ~,; ,.~, .. . .
J ~,), , J .
the current invention.
The prior art contains certain examples of application programs which, on one level, exhibit some of the behavior of a notebook. "Hypercard" from Apple Computer and "Current" from IBM both can present an image of a notebook with page-turn buttons, tabs and an index. The tabs and index entries can be selected with the mouse cursor to turn to the corresponding pages.
A major difference between these existing application programs and the current invention is that their notebooks are static objects contained 1 0 within a single application program. The number of pages, page contents,buttons, tabs, and index entries were laid out by the designer of the application, and do not change in normal operation. In contrast, the notebook in the present invention is designed to control an indefinite number of documents created by a variety of diverse application programs 1 5 such as word prooessors, nurneric spreadsheets, drawing programs, address lists, FA~C viewers, and computerized forms. A notebook contains an arbitrary and changeable number of these documents, one per page, which need have no knowledge that they are embedded in a notebook. The notebook in the present invention acts as the "job control language" for a 2 0 collection of programs and their associated files. The contents of the table of contents and index change as documents are created, moved or deleted. The visible bookmarks or tabs can be enabled on a document-by-document basis.
In addition, any number of pages in the present invention can be "removed"
from their home position and "floated" over another page and then sent 2 5 home. No known work in the prior art exhibits this set of features. Or, to state it another way, although the present invention could be used to build something mudl like "Hypercard", the converse is definitely not true.
Summarv of the Invention 3 0 The present invention indudes methods and apparatus for controlling a computer. The computer presents the behavior of a multi-page notebook that resides on a bookshelf containing other notebooks and accessories.
wO 92/081gs ~ Pcr/uss l /oxo7, ~a9~2~
Features include visible numbered pages, page-turn effects, tabbed bookmarks along one or more edge, "floating" pages temporarily removed from their notebook location, a table of contents, contents of sub-sections, and an index.
A "Document Model" is presented to the user in which the existence of an underlying computer program implementing the behavior of a document is hidden.
Each page in the notebook contains one document which may contain arbitrary amounts of data. These pages are analogous to the combination of data files and application programs in other computers, and may be compound docurnents containing different types of documents such as text, drawings, F.AX images, or specialized forms.
The user has complete control over the ordering of his documents by placing them where he wishes in the notebook. In addition, arbitrary linkages rnay be made between docurnents by easily-created "link buttons"
placed in one document and tied to an object in another. When such a button is selected, the page of its target is turned to. The construction of compound documents is supported and implemented by t} e computer's operating system, and does not require an application program to contain any special code to support sub-documents or to be a sub-docurnent.
2 0 The notebook metaphor presents a virtual object which everyone is familiar with and can use: a book. Components such as the table of contents, page numbers, and page tabs are instantly recognizable and can be manipulated by a novice user. The match to a real-world object is even stronger when this metaphor is implemented on a "note pad"-style computer 2 5 which uses a stylus as a pointing device directly on the screen.
Such a computer is described in detail in another United States patent application filed by applicants herein, namely Agulnick et al., "Control of a Computer Through a Position-Sensed Stylus," Serial No. 07/610,231, filed October 31,1990. That application, including its appendices, are incorporated 3 û herein by referenoe. The appendices are all from GO Corporation of Foster City, California, and are en'dtled:
I. Architecture-Parts 0 to S
- , .. ...- ... . . .
-. . . . . . ..
' . ~ ; :. ~ : . ,, - . , . ~ . .
~ , - . . . ~.
- : , WO ~2/08199 PCI/IJS91/08077 ~ ~ J !), , J J
- IL Architecture--Parts 6 to End m. Application Developer's Guide IV. API Reference.
Another copending application which complements the disclosure 5 herein is the United States patent application of Hullender entitled "Method For Pattern Recognition," Serial No. 07/607,125, filed on October 31, 1990.
That application is also incorporated herein by reference.
The notebook of the present invention allows the user to impose a variety of structures on his documents. The structure can be as simple as a 10 list, or divided into sections, or even a large hierarchy with nested sections.
This structure itself can help the user remember where documents are, and can place related documents together, and also supports a variety of retrieval paths. The user can flip from page to page, or turn to a specific page by selecting its tab or entry in a table of contents or index. The table of contents 15 and index provide complementary browsing mechanisms, one ordered by page and one by name.
Because the functions of embedding and the construction of compound documents and link buttons are supported across-the-boa;rd in the operation systern, these functions are available in all application programs. In addition,2 0 the operating system is free to use these functions to build its own user interface.
The entries in the table of contents and index along with page tabs are all forms of link buttons. The bookshelf, notebooks, table of contents and ` documents are all independent programs embedded one within the other.
2 5 This support for ernbedding is built of robust, well isolated operating system-level objects such as independent tasks, file directories, and display windows.
The present invention implements an Operating System and a Graphical User Interface (GUI) on a computer's graphical display. In a GUI, the computer presents a simplified simulation of some portion of the real 3 0 world. Objects within the computer such as files, directories and programs are represent on the display as small pictures or icons. These icons are selected or moved around the screen by using some pointing device such as a .' ~ ' . ~
-. . . ;
-WO 92/08199 6 PClt~JS91108077 mouse or electronic stylus. The goal is to provide the user with an intuitive feel for the behavior of the objects stored by the computer. This is done by allowing for their direct manipulation with the pointing device. Objects may be "opened", "closed", or "dragged" onto/into other objects.
The most common pointing device is a mouse with one to three buttons. A cursor on the screen tracks the movement of the mouse.
Commands are issued by moving the cursor over some area of the screen and pressing one of the mouse buttons. Pressing and releasing a button is a "click" which can be extended to double and triple clicking. Holding a button down and moving the mouse generally is used to drag the object beneath the cursor.
An electronic stylus whose position is sensed by a digitizer can also be used as a pointing devioe. In addition to the actions available with a mouse, the stylus adds gestures. For example an object can be deleted by drawing an 1 5 'X' over it. If the digitizer is integrated with the display, then the cursor may ` be dispensed with, because the stylus can point directly to displayed objects.
The present invention can thus be used with a mouse, stylus or other pointing devices.
The implementation of the actual "notebook" GUI software can be 2 0 based on any one of many prior inventions. Information in this area can be found in Inside Macintosh,Fundamentals of Interactive Computer Graphics, SmallTalk-80, The Language and lts Implementation (all from Addison-Wesley Publishing Co.), "The X Window System" (ACM Transactions on Graphics, Vol. 5, No. 2, April 1086, pp. 79-109) or "Andrew: A Distributed Personal Computing Environment" (Communications of the ACM, March 1986). These referenoes are incorporated herein by reference.
Brief Description of the Drawin~s Figure 1 shows an initial display of the notebook-metaphor computer of the present invention.
Flgure 2 shows the opening of a window related to an icon on the bookshelf.
,,, ', .
, . . . . . . . . . . . . . . . .
.-. . - : . ,: : - . .: : . - .. .
WO 92/08199 ~ PCr/USYI/08077 r~ ~, J <J ..,; `J
Figures ~5 show different types of icons available to the user.
Figure 6 shows the re-sizing of the displayed main notebook Figure 7 shows a document comprising a drawing.
Figure 8 shows a document comprising text.
Figure 9 shows the document of Figure 8, solled down to a lower portion of the document.
Figure 10 shows a docurnent utilizing various features common to all documents in the notebook-metaphor system.
Figure 11 shows the document of Figure 10, with the document menu 1 0 displayed.
Figure 12 shows the document of Figure 10, with an option sheet floating above it.
Figure 13 shows the document of Figure 12, with the displayed list of accessible sheets.
1 5 Figure 14 shows the document of Figure 13, with the access sheet floating above it.
Figure 15 shows a text docurnent with an embedded trawing document.
Figure 16 shows the text document of Figure 15, with the border of the 2 0 embedded document disabled.
Figure 16 shows compound documents built from other documents.
Figure 17 shows a text options sheet floated above a document.
Figure 18 shows the document of Figure 17, with an Edit menu floated above it.
2 5 Figure 19 shows a Table of Contents with a help notebook floated above it.
Figure 20 shows a contents notebook.
Figure 21 shows the contents notebook with the View menu displayed.
Figure 22 shows the contents notebook of Figure 21, with one of the 3 0 entries expanded.
Figure 23 shows the contents notebook of Figure 21, with all sub-sections expanded.
.
~ '. - .
- : ,.. .. : . - .. - - : . .
, .. . . ... . .
.. . ~ . .
.. . . .. . .
. . .
wo 92/08199 8 Pcr/US91/0807l 2~3~
Figure 24 shows a Table of Contents with the Show menu displayed Figure 25 shows the Table of Contents of Figure 24, with the hidden page tabs displayed.
Figure 26 shows a contents page with a FAX page floating above it.
Figure 27 shows the FAX page of Figure 26, expanded to full size.
Figures 28 and 29 show the r~sizing of the FAX page of Flgure 26.
Figures 30 and 31 show the repositioning of the FAX page of Figure 26.
Figure 32 shows the contents page with the Edit menu displayed.
Figure 33 shows the contents page with the Create menu displayed.
1 0 Figure 34 shows the contents page with the Sort menu displayed.
Figure 35 shows the index page of the notebook.
Figure 36 shows a document with numerous link buttons linked to headings in the document.
Figure 37 shows a document with a link button embedded in the 1 5 document.
Figure 38 shows a lin`k button from one document embedded into another document.
Figure 39 shows the floating of one document abo~re another due to double-clicking of a link button.
2 0 Figure 40 shows a link button linked to a word in the help document.
Figure 41 shows the manner of storage of documents in the file system of the invention.
Figure 42 shows a compound document with embedded documents.
- Figure 43 shows the editing of an embedded document.
2 5 Figure 44 shows the result of the editing shown in Figure 43.
Figure 45 shows the in-line emplacement of an embedded text document.
Figure 46 shows an ernbedded drawing document with its borders displayed.
3 0 Figure 47 shows the document of Figure 47, with the Access menu displayed.
Figu~e 48 shows the result of remo~ring the borders from the embedded .. :
: ..... , .. . .. . , ... .. .. . . ~ ; ... .. .... .
. -- ., . .
.. - . . . . . . ~ .
- -. : :: , ~ , - . . ' ~ . . . -:
.. . . . , . ~ .
... . . . . . . - . . .
~) ~; " ;` - ~,~
document in Figures 46 and 47.
Description of the Preferred Embodiments Figure 1 shows a typical initial display or screen of the notebook computer. The upper portion of the display shows the "table of contents"
display 100 of the m~un notebook 10. The lower portion 110 is a "bookshelf"
of icons which represent notebooks 120, various accessories 200 or any other object that the user wishes to place there, such as a link button 230.
The bookshelf is the highest level "document" in the system, and all 10 other objects or their containing documents are found there. Clicking or tapping (as with a stylus) on an icon 150 on the bookshelf will open a window 240 for that item. When an item is open, its icon is presented as a gray shadow 250 of the nonnal icon. ~he main notebook resides on the bookshelf but is open, and therefore its icon 260 presents the shadow effect. The objects 1 5 (such as icons and buttons) ernbedded within the bookshelf are available tothe uær of the computer from any of the pages stored by the system.
Some of the other items shown on the bookshelf 110 are: the '~Ielp Information" notebook 120; the document contai~ung users preferenoes 130;
the stationery notebook 140, the acoessories container 150, the in and out 2 0 boxes 160 and 170, respectively; the application program installer 180; the "on screen" keyboard 190; the disk browser 200; and the installed devices notebook 210. The uær can set a preference controlling the format of the icons on the bookshelf. The possible formats are large icons as shown in Figure 1, large icons with titles 270 (see Figure 3), small icons 280 (Figure 4), or small icons2 '5 with titles 290 (Pigure 5).
The main notebook 10 floats over the bookshelf. As shown in Figure 6, the lower edge 300 of the main notebook 10 can be dragged up or down to reveal more or less of the bookshelf 110. This re-sizing is done by dragging the box 310 at the center vertically with the pointing device.
3 0 A notebook is composed of a series of pages numbered sequentially by the software of the invention with the first page (the table of contents) being page one. A page is a document of some type; i.e., each document is assigned - . -- - . .. ; , - . . . ; :- ,.. .
~. ' ' ,' - : - : ~ . , ,::. . ,'.. . - . . , . . . , . : .
- - ":. : ' , ' ~ ' .' .' - . . ': ' ., - . . ~, .
: ~
Wo92/08l99 1 ~ Pcr/ussl/nxo77 ;, ;l3 ~
to a page by the program of the invention. The table of contents .md other su~section dividers are all types of docurnent. Other types of docurnents are text (Figure 8), numeAcal spreadsheets, drawings (Figure 7), FAX images (Figure 27), charts, or compound documents (Figure 16) built of other documents. Thus, each document is created by at least one application, and different docusnents are sequentially ordered by the operating system of the mvention.
Each page in a notebook is a view into some (possibly larger) document. Thus a page could represent a 30 page text docurnent or a 6 by 9 foot drawing. For documents where the window is smaller than the full document, any portion of the document can be viewed by scrolling the document horizontally or vertically through the page window, using scroll margins 3ZO shown in Figure 7. A notebook can be made to rnimic the behavior of a physical book by sisnply restricting the contents of each page to 1 5 what will fit in the visible portion of the page. The Help notebook 790 (see Figure 19) is an example of this; as seen in that figure, the Help notebook is organized as a plurality of pages with a sequence and numbering which are independent of the sequence and numbering of the Sample Notebook ~over which the Help notebook is shown floating).
2 0 Figure 8 shows a text document with a scroll margin 340. Documents can contain vertical and horizontal scroll margins in any combination; In general, scroll margins are made up of four elements: a slider 350, selection area 360, and the direction asrows 370 and 380.
The position of the slider 350 shows the position of the visible portion 2 5 relative to the whole document. This visible page display (or doc~nent display) region on the screen can be moved by using the pointing device with the selection area or direction arrows. A click on a direction button moves the view of the window into the document one line in that direction.
Tapping or cliddng within the selection area causes the region of the 3 0 docurnent which corresponds to the location of the tap to be displayed in the window. The slider may be dragged up or down to a new position 390, causing the display of the corresponding region 400, as shown in Figure 9.
I
- - . - - .:.. ~ . .. .
.
- .
.
~ . . .' - ~
wc) 92/08199 1 PC r/lJs9l/08~)7 /
u Figure 10 shows features such as the page title bar 410, page number 420, page turn buttons 430 and 440, and optional visible page tabs or bookmarks 450, which are features common to all pages presented in the notebook metaphor of the invention. The page names or titles, page 5 numbers and tabs serve as page identifiers associated with the sequentially ordered plurality of pages. The page tabs are available to the user from any page, and are ordered in accordance with the sequential ordering of the pages;
similarly, the tabs are automatically reordered to reflect any reordering in thesequenoe of pages.
1 0 As discussed below relative to embedding, a page identifier (such as "Org Chart" in Figure 25) may refer to a page which includes other pages embedded within it.
The menu bar 460 occurs on many pages in the notebook, and includes - one or more buttons 470. Typically, the activation of these buttons displays 15 sub-menus of related commands or options.
The "Document" menu 480 and "Edit" menu 490 are also available in all documents. An optional "cork margin" 500 may appear at the bottom of the page. The cork margin is essentially a private bookshelf for each document (or page) and may contain embedded documents, accessories or 20 link buttons 510.
The Document menu 520 shown in Figure 11 provides access to commands which operate on the whole document. The "About..." button 530 causes a stack of sheets 590 (see Figure 1V to float over the current page.
These "option sheets" display and control attributes of the documen$ as a 2 5 whole. The three buttons 600, 610, and 620 at the bottom of the sheet 590 are used to apply the options specified in the sheet and close the stack.
Other sheets in the stack are accessed by clicking on the current sheet's title 640 and then choosing from the displayed list 660, as shown in Figures 12 and 13. The "Document" sheet 590 provides access to such attributes as title, 3 0 author, comments, s~ e and access dates. The "Access" sheet 670 enables or disables the various controls that surround the body of a document. These options are generally used when embedding one document into another to - -~ .:. , . . -- . --.
- . : ,,: ~
- - . . - . : -- . . :
. - - . . . : .
- .
WO 92/08]99 PCT/US91/081)77 2 ~ ~3 ~
cause the embedded document to blend in with the surrounding main document.
Figure 15 shows a text document containing an embedded drawing document 680 with its surrounding border displayed. Figure 16 shows the 5 same docurnent with the border of the embedded docurnent disabled, seamlessly merging the two documents. Like floating pages, option sheets can be dragged using the title bar 650 and r~s~zed by dragging the "handle"
630 on the borders (see Figure 12).
Option sheet stacks are used throughout out the present invention to 1 0 control the attributes of many types of objects. For instance, the fonts andsizes of letters in a document's text can be controlled through a text option sheet 690, shown in Figure 17. Option sheets are available for such diverse objects as letters, paragraphs, pages, geometric figures, spreadsheet cells, graphical images, icons, or whole documents.
Referring to Figure 11, other commands on the "Document" menu will print the docurnent (command 540); control the forrnat of printing (command 550); send the document to a remote computer or device (command 570); export the document to a foreign file fonnat (command 570);
or revert to the previous version (comrnand 580). The exact contents of the 20 "Docurnent" menu is dictated by the type of the document, but the "About"
command 530 is always available.
The "Edit" menu 720, shown in Figure 18, contains commands which apply to objects selected from within the document. A selection is generally made by clicking or tapping on the target, or by dragging the cursor through 2 5 many objects. Por example, a text document is biased towards operations on words: double-clicking over a leKer selects the entire word. Many words could be selected by dragging the cursor or pointing device through the desired words.
In text, the selection is indicated by a gray background 730. Other types 3 0 of objects can use different indications, such as the small square handles 330 displayed by geometric figures, as in Figure 7.
The Edit menu 720 shown in Figure 18 typically provides commands to - : .. . - . :: .
~. ... ~ . .
. ~: . , . - . ~
... ,.,.,.,.. ,. - ` , ~: , . , . : ..
. ................ :
- . .: , ... ,.~,~
-. ,, :~
.
~ '' .
~ ~ 3 ~, ~ " ~j move (cornmand 740), copy (cornmand 750) or delete (cornmand 760) the selection, as well as to access (command 770) the option sheet stack that pertains to the selection. Other commands 780 specific to the objects found in different document types are located found on the "Edit" menu.
It is also possible to move or copy text within a document or to another document by using gestures. A target object is first selected by pressing the pen on the selection until an indication of the selection occurs, such as highlighting or an animated line surrounding an icon which shows the selected object (or a portion of it, if the selection is text longer than a few words). In a "move", the selected wording is actually moved (which is to say, it is deleted from its original location); in a "copy", a copy of the selected wording is moved.
The icon is then dragged to another location within the document or page being displayed, or to another location within the notebook to which the user navigates. In the latter case, the user may turn to a different page, in which case the icon will follow the page turn, and then "drop" or release the icon (and henoe the selected information) by lifting the pen at the new ' location. For instance, ~e user may turn to the table of contents and drag a target object such as a docurnent to a location on the table of contents.
2 0 As indicated above, this prooedure may be used either to drag a copy of the target, or to move the target, to the new location. The move and copy operations may be distinguished either on the basis of gestures, or by using the Edit menu. Thus, a move operation may be carried out by tapping the target three times in quick sucoession, followed by navigation to the new 2 5 location, dragging the targe~ (or the icon thereof) to the new location, and dropping the copy there. Alternatively, the target may be selected, followed by a selection of "Move" from the Edit menu, and then dragging and dropping a copy of the target thereto. af an entire document is moved, as noted earlier, its page number c`hanges and the operating system of the invention 3 0 automatically resequences that documents and the other documents is the sequential order of documents.) A copy operation, on the other hand, may be a press-hold-drag-drop gesture, or a select-drag-drop gesture coupled with the ~, . - . .......... . -, ' , . : - . . .
. .
WO 92/08199 PC-r/U~91/08077 choice of "Copy" from the Edit menu after selection of the target.
Some specialized documents such as "Help" are better presented as notebooks separate from the main notebook. This separate-notebook format allows a page from the "Help" notebook 790 (see Figure 19) to be referenced as it floats above a user's document 800. Such independent notebooks support all of the features of the main notebook described above. A user might also organize different sets of documents into separate notebooks.
The "Contents" page of a notebook, shown in Figure 20, contains entries 810 for all top-level doruments contained within that notebook. Each 1 0 entry includes an icon 820, a name or title 830, a button 840, and a page number 850, all of which are automatically generated by the system of the invention, and which provide direct access to the various associated documents.
The contents page is itself the top-most instance of a docurnent of type "Section". ~he entries 860 whose titles are bolded (Samples, Staff, etc) are also "Section" doalments which contain sub-sections of the notebook, and whose first pages constitute subsections of the Table of Contents. Each section includes a tab (such as the tab for 'Widget Inc" shown in Figure 20), which both identifies the section and includes a subset of the tabs of the notebook.
2 0 The tabs are automatically assigned the names of the pages which they identify, but can be renamed. If a document title or name is changed, and the associated tab has not been edited, the tab is likewise changed. However, if the tab label is edited and the document or page name is subsequently changed, this new change is not automatically implemented in the tab name.
2 5 The entries 870 whose titles are in a font of normal weight represent documents of other types. A page is selected by clicking on its ti~e, and its selection is then indicated by a gray background 880 on the title.
Referring to Figure 21, an option for an expanded view (command 890) or collapsed view (cornmand 900) can be selected from the '~iew" menu 930.
3 0 In an expanded view (fig. 22) the nested contents 940 of the section will bedisplayed. Double-clicking on a section narne will also expand or collapse that section, depending on its state.
~ . .
WO 92/08199 ~ ~ PCI`/l'~ 807/
S ,, ~, If "Expand" or "Collapse" are chosen when no page is selected then all sections are affected. Figure 23 shows a portion of the table of contents with all sub-sections expanded. Note that a scroll margin 950 has been automatically added because the expanded list carmot be completely displayed in the window.
The pages of a notebook may be accessed in many ways. Adjacent pages may be turned to by dicking on the "next page" 440 or "previous page" 430 buttons, shown in Figure 10. Clicking on a document's page number 850 (see Figure 20) in the table of contents or icon 820 will turn to that page. Pages may also be accessed by selecting them and choosing "Bring-to" 920 or "Turn-to" 910 (see Figure 21) from the "View" menu 930.
Page tabs 960, shown in Figure 24, are visible for pages where a page tab has been enabled and can be clicked upon to turn to that page. The table of contents always has its page tab as the top-most tab 970. A page's optional tab 1 5 is controlled by a check-box 980 which is made visible by selecting the '~ab Box" option 990 from under the "Show" menu 1000. When there is not enough room to display all the page tabs the lower tabs 1010 will be stacked.
In order to see these hidden tabs a special view 1020 (see Figure 25) is available which shows all tabs. This view is obtained by flicking to the left on any of 2 0 the hidden tabs. Any subsequent action will again collapse the tabs, and tapping on any given tab will turn to ~e referenced document.
The "Show" menu 1000 (shown in Figure 24) in a section allows individual control over what information is shown for all listed documents.
Each item may be separately toggled by tapping on it. "Icon" 982 controls the 2 5 display of the small icon defined for each document type. "Type" 984 when enabled will display the name of a documents type. "Date" 986 controls the display of the document's last modification date. "Ske" 988 controls the - display of the documents' filed (i.e., inactive) size. '~ab Box" 990 will display the page tab check boxes when enabled.
3 0 Where a single cliclc causes a page to be turned to, a double click willcause that page 1030 to be floated above the current page. The turn-to/bring-to meaning for clicking and double-clicking is used in many places .
.. . . ...
- -.. . .
.
- ~ -wo 92/08l99 1 6 Pcr/ussl/08077 7'.~
throughout the notebook The icons 820 and page numbers 850 in a section document, the tabs 960, the page numbers 1290 in the index document (see Figure 35) and "go to" link buttons 230 all support this behavior.
A floating page can be sent back to its home location by clicking on the triangular button 1040 (see Figure 26) located at its upper left corner. Floating pages can be expanded to the full size of the display, as shown in Figure 27, byan upward stroke in the title bar 1050, and can be reduced to their original floating size by a downward stroke on the title bar 1080 of the full size page.
In addition, floa~dng pages can be re-sized through the "handles" 1060 1 0 and 1070 at the lower right and bottom of their frames, shown in Figure 26.
The boundary 1090 of the floating page can then be shrunk or stretched by dragging the box to the desired position. Figure 28 shows the expanding boundary 1090, and Figure 29 shows the result. In a similar fashion the location of floating pages can be changed by dragging an outline 1100 (see 1 5 Figure 30) by means of the title bar to the new location 1110, shown in Figure 31. Floating objects maintain ~eir positions as the underlying page is turned.
The "Edit" 1120 menu of a section document, shown in Figure 32, is used to move ~command 1130), copy (command 1140), delete (cornmand 1150) and rename (cornmand 1160) individual pages.
2 0 New pages are added to the section by choosing a document type 1190, shown in Figure 33, from under the "Create" menu 1200. The new page is placed after, and in the same section as, the current selection. As shown in Figure 34, the pseudo-entry "--Empty--" 1180 may be selected to place a page in an empty section 1170. If there is no selection, then the new page is placed at 2 5 the end of the list of pages.
Although section documents normally display their contents in page-number order (as in the TaUe of Contents), the "Sort" menu 1210, shown in Figure 34, can provide other orders. Entries wi~in a section can be listed by page (command 1220), name (i.e. alphabetically: command 1230), type 3 0 (command 1240), date (i.e. chronologically: command 1250) or size (command 1260). Thus, cornmands in the Sort menu let the user rearrange the Table of ; Contents by criteria similar to the Show menu. The default display order is .
" ~' . ~ ' '' ' ' ~
' ' ' ~ : . ': ' WC)92/08199 1 ;7 PCJ/US91/()%077 ~ ~j .J ~J i .'J ~
the order in which the docu~nents appear in the notebook.
Sorting docurnents using the Sort menu rearranges their order in the Table of Contents, but in a preferred embodiment does not affect their actual sequential ordering as governed by the operating system. However, the creation, deletion, insertion and moving of a document (l.e. transfer from one location to another) do cause the operating system to automatically reorder the pages to reflect such actions.
As in many books, an index 1270 (shown in Figure 35) may be found at the back of a notebook. Like its real-world counterpart, the index is an 1 0 alphabetical list of keywords or phrases 1280, and of the numbers 1290 of the pages on which they occur. The system of the invention automatically creates the index, and automatically modifies it as documents are created, moved, modified and deleted, to reflect the new sequential ordering of documents as managed by the operating system.
1 5 Keywords that appear on multiple pages list all occurrences 1290. The initial set of entries includes the titles of all the documents in the notebook.The user may add keys 1300 and 1310 from any words 382 and 384 (Figure 8) in any document in the notebook. Pages can be accessed by clicking or double-clicking on the page number to turn-to or float the page, respectively.
2 0 link buttons allow the user to place ties from anywhere in the notebook to any other object in another document. Figure 36 shows a very useful application of link buttons. The main document is a long, frequently-referenced report. In this case, link buttons 1320 have been placed on the document's cork margin 1330, and these buttons have been tied to some of 2 5 the mapr headings 1340 in the report. Clicking on any of the buttons 1320 causes the document to scroll the target to the window.
Such a link button could also be placed: in the body of the document, as shown at numeral 1350 in Figure 37; or on the bookshelf 230; or in another document ~see numeral 1360 in Figure 38). No matter where the button is 3 0 placed, clicking on it will cause the target to be turned to, while double clicking will float the target 1370 as shown in Figure 39.
Thus, generally speaking one docurnent may be assigned to a first page, .:. ~
- . .
WO92/08199 1 a PC~/US91/1~8077 s ~
and another document may be assigned to a second page, and the link button is aeated and assigned to the first page, i.e. the page with the buHon, which isdisplayed in the document display region. The ~ink button is, in the example above, utilized while the first document is displayed in the document display 5 region, unless it resides on the bookshelf, in which case (like all other items on the bookshelf) it is always available regardless of the document which is displayed or the application which is in use.
It is quite easy to use link buttons to create a "hypertext" document.
Figure 40 shows a link button 1380 on ~e bookshelf tied to a word 1390 on a 1 0 frequently-referenced page in the Help notebook 1400. When the button is pressed, the Help notebook will open and turn to the linked page.
From the above examples, it is seen that link buttons may either cause the target document (such as document 1370 in Figure 39) to be displayed in a second document display region, i.e. one that does not precisely coincide with 15 the first document display region, in which the linking document is displayed. Alternatively, utilizing the link button may cauæ the target document to be turned to, i.e. displayed in the first document display region, replacing the contents of the first document.
20 Embedding And The Implementation of the Notebook Metaphor Embedding is a fundamental concept in the implementation of the present invention. For the purposes of this application, ernbedding can be defined a~s the construction of compound documents by the nesting of autonomous documents within another document. This nesting can be 2 5 carried out to any level. A simple example of this is in Figure 42, showing the construction of a document containing some text 1470 and 1480 and a diagram 1490. This document is constructed by embedding a drawing document into a text document. Without support for the embedding of independent disparate documents, the underlying text editor program would need to support the 3 0 functions of a drawing program. With embedding, however, a user can construct compound documents from any available document, where each document may include an independently ordered sequence of pages.
.
.
.
. . .
wo 92/08199 Pcr/~ls9l/0807/
l C!
~ r' r~
Moreover, as shown in Figure 42, the embedding document may be created by one application, while the embedded document is created by a second, different application.
The notebook takes this structure to its logical end: the entire system is built as one all-encompassing document. The root document is the bookshelf. All objects on the bookshelf are embedded in the bookshelf, including the main notebook. The sole document directly embedded in the main notebook is the table of contents document. The docurnents embedded in the table of contents are chosen by the usen They may be of any type of 10 documents, including multiple documents, compound documents or section documents which contain other documents. The embedded document is preferably displayed in its position as embedded in the embedding document in the document display region, or an icon of the embedded document may be displayed there instead. The text surrounding the embedded document or 15 its icon preferably flows or wraps around it, filling in the surrounding area in the document display region.
The notebook metaphor hides the existence of the computer programs which implement its behavior and that of the different types of documents.
Other, earlier systems use a model in which a program is executed and can 2 0 load, modify, and save dah f;les of an acceptable format. In the present invention, by way of contrast, all the user needs to see are documents. The execution of the correct program and the loading of a specific data file normally occurs when a document is turned-to, floated or opened from an icon. Thus, an embedded document may be displayed and modified by 2 5 floating it in a display region above the display region in which the main (embedding) document is displayed; or it may be modified in-place, i.e. where it reddes in the embedding document. If it is floated, then it may partially cover up the display of the embedding document, but dces not replace it in the first document display region.
3 0 As a general rule, a document may be floated over aI~ underlying document appearing in the document display segion in at least two manners:
first, such that the floated document remains in its own display region when - -- - . , .
- . : . ~ - :. ,. ~ :
- . . . . - . . - - . .
- . .
. - . ~ . .. . .. - .
.
- . - , . . . - : --. .~ - . : :
.. - - -- ~ . . . : . --, . . ...
WO92/08199 2 o Pcr/us91/08077 the user turns away from the underlying document (such as to yet another, i.e. third, document); and second, such that, when the user turns away from the underlying document (i.e. to a third document), the floated document ceases to be displayed.
An identifier that uniquely identifies the type of the document is saved with the document's filed data when it is eated or suspended. This identifier is used to select the appropriate program code to execute when the document is made active again.
The present invention implements embedding as part of the main 1 0 operating system software, making embedding available to all application prograrns that execute within this "application framework". An application program need contain no code to deal with accepting embedded documents or being embedded itself. The application framework provides default behavior for embedded documents, but programs can override this default if desired.
- 15 The application framework is implemented on top of a preemptive, multi-tasking operating system which supports shared code, a tree-structured file system and a windowed graphics system. The support for embedding is built from the common "Operating System"-level objects. A non-compound (or simple) documerlt is a separate task executing the shared code for that type2 0 of document. When the user turns away from a document's page or closes its window, the task is suspended and its state is saved in a sub-directory whose position in the directory tree corresponds to the position of the document's nesting within the bookshelf. This filed copy of the task's state includes more than just thé data that makes up the document. Enough information is saved 2 5 such that when the task is re-started, the displayed page or window will appear exactly as it did when it last was displayed. To do this, the state of the window is also filed with the state of the document.
The application program displays its output and acoepts pointing-devioe input through a window whose location~and size are controlled by its 3 0 parent application or (by default) the application framework. Thus, the application framework isolates an application program within its own task, directory and window. This isolation builds a very robust system without . ., . ; ~ ~ . . ' , ; ' : . - .
. :. .
- , ':: ' ';
WC)92/08199 2 1 PCI/lJ~i91/OX077 ?
buriening the user with an arcane command language. The user can locate, execute, combine, suspend, and destroy multiple tasks by simply manipulating pages in a notebook.
The bookshelf and its nested documents are cor~tructed of standard components commonly found in tree-structured file systems, such as files and directories. The operations performed on these components (move, copy, delete) are also available in all functional file syste ns. Because of this, it is easy to map the structure of the present invention onto a foreign file system. If this external file system can be shared by multiple machines, tihen a1 0 number of machines implementing the present invention can share and exchange entire notebooks through the shared mediurn.
Consider the document shown in Figure 42. This cornpound document was created by embedding a drawing document 1490 within a text docurnent called "Org Chart". Figure 41 shows how these docurnents are 1 5 stored in the file systern of the present invention. Each endosing document is represented by a directory 1420, whidh contains its sub-documents 1410. At each level a file called "docstate.res" holds the state of the document when it is inactive. The text do~nent "Org Chart" 1420 is embedded in a section document 1430 called 'Widget Inc." whidh is, in turn, embedded in the table 2 0 of contents document 1440. The table of contents is the sole top-level embedded document within the main notebook 1450. At the highest level the main notebook is embedded in the book shelf 1460.
Because each embedded document has its own task which executes when the document is visible, the drawing 1490 is not merely a static image, 2 5 but is a window or ("edit region") into a drawing document controlled by an actively executing program. Thus if '~om Keating" 1510 decides to be known by his middle name (see Pigure 42), his "box" 1500 could be re-sized and his name edited in the edit box 1520 directly in the displayed compound document (where, in this case, the embedded document is created by a 3 0 different application than the embedding document), to produce the revised version shown in Figure 44. To do this, it is not necessary to select the embedded document or switch to another program; that is, when the . . ~ . :. .
- . .
.
.. .: .
. .
- . : . - - . . ': ' :
wo 92/08199 2 2 Pcr~us9l/oxo77 ~,, ` , ` ^ . ., embedding document is opened, the embedded document (here, the organization chart) is automatically opened in an edit region at which it can be edited, using i.s own application (i.e. the drawing application, as opposed to the text application of the embedding document).
In the above example, the modification of the embedded document takes place by means of alterations made with the embedded document in place within the embedding document. Likewise, other actions may be performed on the embedding document which are also automatically executed on the embedded docurnent, such as printing, copying, deleting, 1 0 closing (i.e. ceasing display) and saving, and so on. In the case of printing, an embedded document may be printed by itself by selecting "Print" from the embedded document's "Document" menu, or the entire embedding document may be printed along with the embedded document by selecting 'Print All Contents" from the embedding document's Print Setup option 15 sheet. Yet another alternative is to print the embeddin~ document only with so much of the embedded document as is displayed on screen. In each case, the print action is executed with respect to the embedded document automatically when the embedding document is printed.
The application framework allows applications to tell the ernbedded 2 0 objects when to file their data. This is done automatically, without the user's specific command; thus, when a user turns away from a page, the application framework tells the application to save its instance data, which includes any data in an embedded document. Similarly, when the larger (i.e. embedding) document is printed. copied, deleted or turned away from, the sarne action is 2 5 automatically taken on the embedded document. In the case of printing a document, the d~splayed portion of the embedded document is also printed.
Alternatively, the entire embedded docurnent may be automatically printed along with the embedding document, by selecting Print All Contents from the Print Setup options sheet.
3 0 A~though the application framework provides defa~lt behavior for embedded documents, parent applications can override these defaults. For example, the bookshelf supports its embedded documents in two states: first, .
.. , .... ~ .
WO 92/08199 2 3 PCI`/US91/08077 V ;J _ L " ,J
as icons 120 (see Figure 1) arranged from left to right, bottom to top; and second, as floating windows, as the "help" notebook 790 shows in Figure 19 A notebook causes its "children" to appear as a stacked series of numbered pages with a current page and any number of floating pages visible.
5 A section document presents its children as an indented list 940, as in FigureZ2. A text document places its embedded document 1530 in-line with the text, as in Figure 45. Opening or re-sizing the in-line document 1530 causes the surrounding text 1540 to "flow" (or "wrap around") to fill in to best fit the embedded document.
To produce a "seamless" document the "Access" option card 1558 shown in Figure 47, is opened by using the "About..." command from the "Document" menu 1580. Setting the "Borders" control 1560 to "Off" and clicking on "Apply and Close" 1570 will remove the frame 1550 that surrounds the embedded drawing. This produces the final document shown 1 5 in Figure 48. Thus, there are two border options: the first option displays the border around the contents of the embedded document, and the second option displays the contents of the eInbedded document with no visible border.
Some embedded objects do not need the full capabilities of a document.
2 0 Such "components" alliow for simpler objects with less overhead.
Components do exist in their own window but do not execute in a separate task and file their data in the directory of their containing document. A link button is an example of a component. It contains three pieces of information which, taken together, uniquely identifies the target of the link. First is the 2 5 unique ID of the parent document that contains the target. Second is the ID of the component or sub-documient that contains the target. Third is a unique token assigned by the immediate container to the target object when the link button was created. For example, within text this unique ID is simply the offset of the target character from the start of the text. When a link button is3 0 selected, the operating system uses the document ID to cause the targeted document to be visible, and passes the other two IDs to the document's underlying application program. Thie application program in turn ensures . . . , " - ~ , . . :
;
'' - ,' -, ,. :.
-, - : ,.
wo 92/08199 ;~ 4 PCr/US91 /OX077 3 ~ ~3 'l that the target is visible within the document's current window The normal behavior for an application is to be suspended when its window is closed or not visible. This default can be overridden by the application when necessary. For example the "out box" is a form of device 5 output queue manager. The out box task executes without displaying a window, moving queued data to devices such as printers and modems. An application can also start its own hidden sub-tasks. For example a spreadsheet could run a task to recalculate in the background while the main task handled user interface events.
The nested structure of documents and background execution can also be exploited by specialized "traversal" tasks which need to "walk" through branches of the tree to process all the simple documents in a compound document. Examples of operations performed by such tasks are "search and replace", "bulk spell checking", and printing. These invisible tasks will start a 15 sub-task for each sub-directory in-turn and issue spedalized requests to the executing application code.
:.
.