![]() | ![]() |
| SEARCH |ABOUT |INDEX |NEWS |CORE STANDARDS |TECHNOLOGY REPORTS |EVENTS |LIBRARY | |
| SEARCH Advanced Search ABOUT Site Map CP RSS Channel Contact Us Sponsoring CP About Our Sponsors NEWS Cover Stories Articles & Papers Press Releases CORE STANDARDS XML SGML Schemas XSL/XSLT/XPath XLink XML Query CSS SVG TECHNOLOGY REPORTS XML Applications General Apps Government Apps Academic Apps EVENTS LIBRARY Introductions FAQs Bibliography Technology and Society Semantics Tech Topics Software Related Standards Historic |
General
XML Conformance[CR: 20010402] The notion of XML "conformance" is addressed directly in the XML specification (Version 1.0, REC-xml-19980210) section"5. Conformance," which includes a discussion on "5.1 Validating and Non-Validating Processors" and another on "5.2 Using XML Processors." Subsections:
W3C/NIST DOM Conformance Test SuitesSee the primaryDOM Conformance reference list. OASIS/NIST XML DOM Level 1 Conformance TestSourceForge XMLConf Project[July 31, 2000]David Brownell recently posted anannouncement for SourceForge XMLConf - a 'Conformance Testing Project for XML and Related Technologies', including XML, XML Schema, DOM, Performance, SAX, and XSLT. TheXMLConf project "hosts XML related testing efforts, focusing initially on conformance testing. At this writing, it's in the very early stages ... so if there's something you think needs fixing, please help fix it! It's a good time to join. Notice that all of this software is under the GPL. The first testing effort hosted here addresses XML conformance. It includes test harnesses for Java (with SAX/SAX2) and for JavaScript (with DOM/COM). The second such effort is currently in its early stages, and addresses XML Schema conformance. Other projects discussed include DOM testing, performance measurement, XSLT conformance ... the whole gamut. Basically, if it's an XML related technology and there's enough of a standard API that an automated harness could usefully compare different implementations, it could fit in here. The intention here is provide a home for open, public, collaborative development of harnesses and test cases for testing XML (and related) processors. It complements the corresponding efforts of W3C, NIST, OASIS, and many others. . ." Principal collaborators named in the announcement include David Brownell, Curt Arnold, Joe Polastre, and Richard Tobin. W3C XML Conformance and Quality Assurance[May 04, 2001]"The Matrix of W3C Specifications." From the W3C"Conformance and Quality Assurance" resource section [Conformance and Quality Assurance Activity]. Maintained by Karl Dubost. For each namedW3C specification, the table (matrix) includes information on the specification status (Rec, PR, CR, WD, etc) and URL for an [existing] QA Log, online Validator, Test Suites, W3C Notes / Tutorials, and Conformance section. [February 07, 2001] W3C Workshop on Quality Assurance [and Quality Assurance Activity]. The World Wide Web Consortium (W3C) announced a'Workshop on Quality Assurance' at W3C. Registration is open through 28-March-2001 for this W3C Workshop on Quality Assurance, to be hosted by NIST in Washington, D.C. USA, on April 3-4, 2001. Participants can share their understanding of Web QA tools, conformance activities at W3C, and discuss a potential new W3C QA Activity. Position papers should be submitted to the Workshop Chairs by March 16, 2001. The main objective of the workshop, identified in thecall for participation, "is to have W3C, its membership and the Web community involved in QA at large to share their understanding of the state of affairs for Web QA tools, technical and business practices and conformance activities at W3C or related to W3C specifications. Furthermore, as we're planning the start of a new W3C activity, one of our goals is to get feedback on the best course of action within W3C that would improve the quality of W3C specifications implementation in the field over time (i.e. what will be in the charter of this activity)." See theQuality Assurance (QA) Activity Proposal (draft). "Besides the shape to give this new potential W3C QA activity, there are several areas of interest related to Quality Assurance and Conformance of W3C technologies that we would like to hear about at the workshop: (1) experience in validation of Web content and documents -e.g., is this CSS page valid?; (2) online testing conformance of user agents - is this multimedia player correctly implementing SMIL1.0?; (3) quality of W3C specifications themselves - wrt conformance statement, tutorial, etc.; (4) conformance testing methodology -e.g., is this CSS page valid?; (2) online testing conformance of user agents - is this multimedia playe test design and components of a test suite; (5) certification/labeling of content, products or services; (6) common framework/harness for running test; (7) coordination with W3C Working Group developing specifications; (8) IPR and funding model." See also theWorkshop Agenda, theBackground Reference Material, and the W3Cmailing list 'www-qa@w3.org'. RXP Output for OASIS Test SuiteRichard Tobin reported: I have recently created a harness to run the OASIS test suite on RXP, and theresults are available online. See: (1)Test results in validating mode, and (2)Test results in non-validating mode. For details on the RXP parser, seethe main web site and thelocal description. RXP is a GPL'd validating XML parser written in C. [tests, cache] W3C XML Schema Test CollectionW3C has also provided for the creation of aW3C XML Schema Test Collection, announced byHenry S. Thompson (University of Edinburgh and W3C; Oriol Carbo, University of Edinburgh and W3C). "Goals and Objectives: The W3C XML Schema Test Collection work aims at coordinating test suites for W3C XML Schema processors created by different developers." The main objectives asannounced 2001-05-02 are: (1) to integrate existing tests for W3C XML Schema processors in a common environment so they can be accessed publicly and shared among developers; (2) to establish a standard approach to test material IPR which meets the needs of both contributors and users; (3) to collect and develop tools to automate the execution and presentation of the test suites; (4) to offer a standard description of tests related to W3C XML Schema processors: [...]; (5) [to provide test descriptions] understandable by a developer without the need to actually view the test file(s) themselves); (6) to offer a standard presentation of test results; (7) to design additional tests and add/regularise descriptions of the existing tests; (8) in due course, to provide an XSLT-based approach to comparing XML representations of the post schema-validation infoset as produced by different processors; we will shortly announce the availability of XML Schemas for both the ordinary Infoset and the PSVInfoset. "The W3C expects to author only a small part of the collection -- we are counting on Member organisations and others to contribute the majority. To offer materials for the collection, please send e-mail towww-xml-schema-tests@w3.org." Note from Henry Thompson: "...thewww-xml-schema-tests@w3.org mailing address isnot a mailing list; it's for potential contributors to use to initiate discussions about contributions. Fordiscussions of testing, I don't think we need a new mailing list; I'd expectxmlschema-dev@w3.org to be used for discussing W3C XML Schema testing..." See thediscussion for the XML Schema Recommendation release. MSXML Conformance[August 31, 2000]"MSXML Conformance Update". By Chris Lovett. From XML.com. August 30, 2000. ['In the past, XML.com has tested Microsoft's MSXML parser for XML conformance with less than glorious results. In this article, Chris Lovett presents the significant improvements made by Microsoft in MSXML in recent months.]' "This article is an update to previous articles by David Brownell on the conformance of the Microsoft XML Parser (MSXML). The July 2000 MSXML 3.0 Beta Release has made a significant improvement in conformance against the OASIS XML conformance test suite. Besides reporting the test OASIS conformance results, this article also reports on the compliance of the new Visual Basic SAX interface included in MSXML 3.0. To run compliance testing on this component I developed a brand new test harness in Visual Basic, which is also included with this article... The OASIS XML conformance test suite is a published set of tests, collected over time from various sources, which measure the conformance of XML parsers against the W3C XML 1.0 specification. It does not include any tests from Microsoft at this time. For my test, I downloaded the updated test suite that David Brownell published in February. This updated test suite takes into account the W3C errata for the XML 1.0 specification. I made two modifications to this suite... seexmlconf-20000821.zip for my updated version of the test suite. . . I used the same ECMAScript test harness that David Brownell published, except for one minor modification. This modification stemmed from the issue of what to do with tests marked 'valid' that have no DTD (document type definition) at all. David's test harness treated this issue in a manner contrary to the design of MSXML. . . MSXML still has some issues to resolve relating to non-existent or malformed unused entities, attribute-value normalization, end-of-line handling, and reporting validity constraints when running in non-validating mode. However, you can see from the following table that MSXML is on a steady march towards 100% compliance." OASIS XML Conformance Subcommittee[CR: 20010305] OASIS (The Organization for the Advancement of Structured Information Standards, formerly 'SGML Open') has set up anXML Conformance Subcommittee under the OASIS technical program, currently chaired by Mary Brady (NIST). The subcommittee will be developing an XML Conformance Test Methodology. [July 30, 1999] In July 1999, the OASIS XML Conformance Subcommittee published anXML Conformance Text Suite (Working Draft 12-July-1999), as described in apublic announcement. [September 16, 1999]"Conformance Testing for XML Processors." ByDavid Brownell. From XML.com (September 15, 1999). ['Not all XML parsers are alike, especially when it comes to how carefully they follow the XML 1.0 specification. Dave Brownell tests Java-based XML parsers and lets you know how closely they conform. This multi-part article evaluates the results of testing a dozen XML processors (XML parsers) against theOASIS Conformance Suite to see how well they follow the XML specification.'] - "One indication of XML's success is that a dozen or so implementations of an XML processor exist. These processors, spanning a variety of programming environments, are at the core of a new generation of web tools that are revolutionizing the dynamic generation of HTML and enabling new types of web applications, including business-to-business data messaging. But are all XML processors doing what they are supposed to do? Will the tools built with those processors create rivers of interoperable messages and documents? Will they create islands of data that can only be used with a single set of tools? This article looks at most of the XML processors available today for use in Java-based XML systems and evaluates how closely they follow the XML 1.0 specification. We will provide you with the hard data from our tests, so you can independently evaluate every claim made in this article and reproduce the results yourself. The results leverage the freely available OASIS XML Conformance Test Suite. While that suite has only been published relatively recently, many of its key components have been well known to the XML community for over a year and a half; they shouldn't come as a big surprise for any implementor. Many of these tests have been used for basic quality testing in a variety of XML processors." For related references, see the announcement for the test suite,"OASIS XML Conformance Test Suite Now Available. International Consortium Provides Set of 1,000 Tests for XML Parsers." [May 12, 2000]"XML Conformance Update." By David Brownell. From XML.com (May 10, 2000). ['David Brownell has updatedthe OASIS test suite to take into account the most recent XML 1.0 specification errata, and run his tests against Ælfred 2, MSXML3, Oracle's parser, Sun's parser, Xerces-J and XP. The open source parsers delivered an excellent showing, leaving Microsoft and Oracle lagging behind.'] "This article is an update to my earlier articles from 1999 that tested the XML 1.0 conformance of parsers. Since then, there has been notable development of the parsers, and updates to the XML 1.0 specification itself. With respect to the dozen Java parsers reviewed last year, there have been a number of interesting changes, both in their conformance and in their open-source status. Microsoft's MSXML parser has also received a significant update, with the release of 'technology previews.'... There appears to be a pause in the OASIS work on XML infrastructure conformance. The ball is temporarily in the W3C's court to address bugs in the XML 1.0 specification that have been reported by its implementors and users. I'm not sure there's enough dialogue in that process: my personal preference would be for the W3C to adopt the IETF approach, and insist on interoperability and conformance testing processes with positive results (and multiple full implementations) before making recommendations." [November 17, 1999]Test Driver, fromDavid Brownell. Dated September 20, 1999 [or later].Pre-Built Source and utilities. "This package is a driver for a set of SAX and XML conformance tests, using databases such as the OASIS/NIST XML Conformance Test Suite. The SAX API is used to access an XML processor (parser), and supports testing of every requirement in the XML 1.0 specification. The database of test cases is used to exercise those requirements. Try using this with any SAX processor, and see how conformant it is! (These use a slightly bugfixed version of the OASIS/NIST test suite. In that suite are almost 1100 tests.) See my conformance testing article at XML.com for more information about the results of this testing. Below are some slightly more up-to-date results..." [November 18, 1999]"Microsoft XML Parser (MSXML.DLL) Conformance." By David Brownell. From XML.com (November 18, 1999). ['More on XML parser conformance: Last September, David Brownell conducted a review of XML parsers for XML.com, testing them for conformance to the XML 1.0 specification. In this follow-up article, he tests Microsoft's MSXML.DLL parser, as found in Internet Explorer 5. The results of the tests gave the Microsoft parser a "pretty good" rating, in the top 25% for conformance. They did however reveal a serious flaw with DTD handling and validation, for which Brownell presents a workaround.'] "Some readers were also confused about Microsoft's Java XML processor, called 'MSXML' in that earlier review. Briefly, Microsoft has had several implementations of XML processor technology. While today one tends to only hear about the latest version of such technologies, they have all been called 'MSXML,' or 'MS XML,' in common usage, by numerous people, including some Microsoft staff. Since the Java processor hasn't been updated in well over a year, some confusion seems inevitable. The Java processor was formally called theMicrosoft XML Parser for Java. I hope that helps to clarify the distinctions between the various packages; the details of the two reviews should also help. The version of the Microsoft XML (MSXML) processor reviewed here is the one that has been bundled with Microsoft's Internet Explorer 5.0 web browser. It can be accessed as 'MSXML.DLL,' and can be redistributed with other software, as part of Win32 applications. Since it provides a COM API, it can be used from JavaScript, C/C++, Visual Basic, and other COM-aware programming languages. It can even be used from Java, but for most Java developers, that support is not particularly useful since it requires using Microsoft's JVM, and does not support the standard SAX or W3C DOM APIs (org.w3c.dom.*)." [December 03, 1999] DOM ECMAScript Test Suite.Mary Brady (NIST, Conformance Testing) posted anannouncement regarding the update of the NIST DOM ECMAScript test suite, accessible viahttp://www.nist.gov/xml/. "Click on DOM Test Suite. This suite includes ~900 ECMAScript tests that exercise the DOM Level 1 Fundamental, Extended, and HTML interfaces. You can view the results using IE5 by clicking on first the category, and then the particular interface. Options are available for displaying the source code, semantic requirements (which are simply axioms we glean from the spec to organize our thoughts), and the actual specification. Please let me know if you find this useful. We are in the process of generating equivalent functionality for the java binding. We are just about finished with the fundamental interfaces, and expect to have a first set, including fundamental and extended available in early January." NIST XML/DOM Conformance Testing. Organized through NIST's Software Diagnostics and Conformance Testing Division [SDTC], underMary Brady. "In order to enable the next level of web applications, we need a flexible method of defining, interacting with, and exchanging data for a variety of domain-specific applications. The Extensible Markup Language (XML) provides a standards-based approach to universal methods for defining and exchanging data. Using XML, one can create customized markup languages for exchanging information within their own domain. In addition, the Document Object Model (DOM) defines ECMAScript and Java bindings for interacting with both XML and HTML data, permitting dynamic creation and manipulation of web pages defined using these metalanguages. Virtually all application domains are looking to use XML and DOM to manipulate and exchange structured information. In addition, XML processors and support for the DOM API are beginning to appear in beta versions of popular web browsers and associated applications. As such, conformance of these products to the W3C Recommendations will permit robust, interoperable solutions." [local archive copy] [February 23, 1999] G. Ken Holman posted anannouncement for the newpublic information page of the OASIS XML Conformance Technical Subcommittee. The XML Conformance Subcommittee was chartered in August 1997, and is chaired by G. Ken Holman (Crane Softwrights Ltd.). According to a presentation by Ken Holman "Report on XML Conformance" at the Seattle XML Conference (XML Developers Day, March 27, 1998), the subcommittee is now "at the requirements collecting stage, [since] many members are currently occupied with XML releases of their own and are learning themselves what are important requirements in this area." See the link to the presentation slides, below. The subcommittee expects to:
[September 11, 1998] Ken Holman prepared a"Report on XML Conformance" for delivery at the XML Developers' Day, August 20-21, 1998. The presentation slides for thisReport on XML Conformance are available in HTML format. [June 04, 1998] A report on the OASIS conformance work was also given by G. Ken Holman at the OASIS Technical Committee meeting held after the SGML/XML Europe '98 Conference (Hotel Sofitel, Friday, May 22, 1998. 9:00 - 15:00). According to Ken's report in this update, current technical work in the conformance committee includes the goal of producing an XML property set (not now a part of the XML 1.0 specification), since the property set might play some role in development of an XML conformance test suite. It is not yet known whether XML conformance might best be expressed in terms of something like RAST (Reference Application for SGML Testing, as defined by ISO/IEC 13673:1995), or the XML property set (and groves), or something else. Work in the OASIS committee (or aligned with it) towards the creation of an XML property set includes contributions by NIST, Alex Milowski, Eliot Kimber, and Gavin Nicol. NIST, through its Software Diagnostics And Conformance Testing Division, is funding the work of an invited expert Tim Boland (t.boland@nist.gov) to assist in the construction of the XML property set and a larger methodology for an XML test suite. The goals of NIST and OASIS in this work are said to be in close correspondence. Ken Holman supervised several working sessions on"XML Conformance" at the OASIS Summer Workshop in San Diego. Current priorities in the OASIS effort include: "1) A simple binary test, whereby a processor is deemed either to conform or not; 2) A test of output, whereby the processor is determined to pass, or not, the appropriate information to the application; 3) A test of resulting properties, whereby the processor is found to produce, or not, a predictable value for the properties." See also:"Report on XML Conformance" at the XML Developers' Conference, Montréal, August 1998. A provisional test suite and a document describing the methodology is to be presented at the XML conference in Chicago, in November 1998. Links:
Testing and Validation Resources[CR: 19990730]
| ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
| |||||
![]() |
Document URI:http://xml.coverpages.org/xmlConformance.html — Legal stuff
Robin Cover, Editor:robin@oasis-open.org