Network Working Group S. Bradner Internet-Draft Harvard U. March 2004 An Idea for an Alternate IETF Standards Track Status of this Memo This document is an Internet-Draft and is subject to all provisions of Section 10 of RFC 2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html Abstract The discussion in the problem working group reached consensus that the current IETF 3-stage standards track, as implemented, is not working. This is a proposal for an alternate, also 3-stage, standards track that I feel better matches current reality. This is a republication of an ID originally published in June 2003 Copyright (C) The Internet Society (2003) 1. Introduction The consensus in the problem working group is that the current IETF 3- stage standards track described in RFC 2026 [RFC2026] is not working as originally intended. The draft problem statement document [prob] says: "The current hierarchy of Proposed, Draft and Full Standard maturity levels for specifications is no longer being used in the way that was envisioned when the stratification was originally proposed. In practice, the IETF currently has a one-step standards process that subverts the IETF's preference for Bradner [Page 1] Internet-Draft Alternate Standards Track March 2004 demonstrating effectiveness through running code in multiple interoperable implementations and compresses the process that previously allowed specifications to mature as experience was gained with actual implementations:" The draft document then goes on to list 4 observations: 1/ few documents actually progress after being published as PS 2/ there is a perception that the IESG raised the quality requirement 3/ in spite of the raised quality requirement, running code is not required 4/ there seems to be a reinforcing feedback loop involved: vendors implement and deploy PS documents so the IESG tries to make the PS documents better The draft document concludes that the 3-stage process is excessive. I disagree that is a reasonable conclusion based on the discussions. Clearly there is consensus that there is something wrong with the current process but I do not think that the consensus extends to saying that any 3-stage process would get the same treatment and this document is a proposal for a revised 3-stage process that I think will meet the needs of vendors and of the IETF. 3. My Observations My observations are somewhat different than those of the problem working group. I think that that, in effect, the standards process has been shifted one place to the left. Vendors implement, and their customers deploy, technology based Internet drafts as soon as the Internet drafts seem to be stable. Thus, Internet drafts have, in effect, replaced Proposed Standard as the first stage of the IETF standards process. But there are significant problems with using Internet Drafts as standards documents. Most importantly, Internet Drafts are not stable. Internet Drafts have short lifetimes with most of them being replaced by new versions or expiring within a few months. If a vendor decides to implement from an Internet Draft they have to be sure that they are implementing the same version of the Internet Draft as the other vendors that they want to interoperate with used. I agree that, over time, the IESG has raised the bar for the publication of Proposed Standard documents. The current level of review, except for not having a requirement for interoperable implementations, is about what I expected the review would be for Bradner [Page 2] Internet-Draft Alternate Standards Track March 2004 Draft Standard when RFC 2026 was published. Thus Proposed Standard has, in effect, replaced Draft Standard as the second stage in the IETF standards process. Having a Draft Standard level of review for Proposed Standard documents has raised the bar for getting working group output published as RFCs such that vendors feel that they must implement from Internet Drafts if they are to make it to marketplace in a reasonable period of time. Very few specifications are advanced to Internet Standard status so that stage has been, in practice, removed from the IETF standards track and Draft Standard has, in effect, become the top rung on the standards ladder. Too few specifications are promoted to the Draft Standard level and a revision of the IETF standards process should try to correct that. 4. Current IETF Standards Track RFC 2026 defines the stages on the IETF standards track as follows: 4.1 Standards Track Maturity Levels Internet specifications go through stages of development, testing, and acceptance. Within the Internet Standards Process, these stages are formally labeled "maturity levels". This section describes the maturity levels and the expected characteristics of specifications at each level. 4.1.1 Proposed Standard The entry-level maturity for the standards track is "Proposed Standard". A specific action by the IESG is required to move a specification onto the standards track at the "Proposed Standard" level. A Proposed Standard specification is generally stable, has resolved known design choices, is believed to be well-understood, has received significant community review, and appears to enjoy enough community interest to be considered valuable. However, further experience might result in a change or even retraction of the specification before it advances. Usually, neither implementation nor operational experience is required for the designation of a specification as a Proposed Standard. However, such experience is highly desirable, and will usually represent a strong argument in favor of a Proposed Standard designation. Bradner [Page 3] Internet-Draft Alternate Standards Track March 2004 The IESG may require implementation and/or operational experience prior to granting Proposed Standard status to a specification that materially affects the core Internet protocols or that specifies behavior that may have significant operational impact on the Internet. A Proposed Standard should have no known technical omissions with respect to the requirements placed upon it. However, the IESG may waive this requirement in order to allow a specification to advance to the Proposed Standard state when it is considered to be useful and necessary (and timely) even with known technical omissions. Implementors should treat Proposed Standards as immature specifications. It is desirable to implement them in order to gain experience and to validate, test, and clarify the specification. However, since the content of Proposed Standards may be changed if problems are found or better solutions are identified, deploying implementations of such standards into a disruption-sensitive environment is not recommended. 4.1.2 Draft Standard A specification from which at least two independent and interoperable implementations from different code bases have been developed, and for which sufficient successful operational experience has been obtained, may be elevated to the "Draft Standard" level. For the purposes of this section, "interoperable" means to be functionally equivalent or interchangeable components of the system or process in which they are used. If patented or otherwise controlled technology is required for implementation, the separate implementations must also have resulted from separate exercise of the licensing process. Elevation to Draft Standard is a major advance in status, indicating a strong belief that the specification is mature and will be useful. The requirement for at least two independent and interoperable implementations applies to all of the options and features of the specification. In cases in which one or more options or features have not been demonstrated in at least two interoperable implementations, the specification may advance to the Draft Standard level only if those options or features are removed. The Working Group chair is responsible for documenting the specific implementations which qualify the specification for Draft or Internet Standard status along with documentation about testing of the interoperation of these implementations. The documentation must include information about the support of each of the individual options and features. This documentation should be submitted to the Area Director with the protocol action request. Bradner [Page 4] Internet-Draft Alternate Standards Track March 2004 A Draft Standard must be well-understood and known to be quite stable, both in its semantics and as a basis for developing an implementation. A Draft Standard may still require additional or more widespread field experience, since it is possible for implementations based on Draft Standard specifications to demonstrate unforeseen behavior when subjected to large-scale use in production environments. A Draft Standard is normally considered to be a final specification, and changes are likely to be made only to solve specific problems encountered. In most circumstances, it is reasonable for vendors to deploy implementations of Draft Standards into a disruption sensitive environment. 4.1.3 Internet Standard A specification for which significant implementation and successful operational experience has been obtained may be elevated to the Internet Standard level. An Internet Standard (which may simply be referred to as a Standard) is characterized by a high degree of technical maturity and by a generally held belief that the specified protocol or service provides significant benefit to the Internet community. A specification that reaches the status of Standard is assigned a number in the STD series while retaining its RFC number. 5. An Alternate Standards Track I would like to propose an alternate IETF standards track with a new stage inserted before Proposed Standard, combining Draft Standard and Internet Standard and retaining Proposed Standard as it has evolved over the years. Part of the problem we have been seeing with getting timely publication of IETF specifications is that once people start implementing the technology it often seems counterproductive to dedicate effort to finishing off the documents. If implementations of Internet Drafts achieve success in the marketplace, as they did with MPLS, it may seem that it is not worth spending time tweaking successive generations of Internet Drafts in order to get something the IESG is willing to publish as a Proposed Standard then, if that achieves widespread success in the market, fiddle with the document again and do the bookkeeping needed to get it published as a Draft Standard. The prerequisites for getting something published as an Internet Standard seem to many people to be fuzzy at best. In addition, the current standards track steps did not do much to encourage early implementations, which are the best way to check to see that a specification is clear enough for implementers to use. Bradner [Page 5] Internet-Draft Alternate Standards Track March 2004 This alternate set of stages tries to encourage vendors to implement specifications and the comments with the descriptions of each stage attempt to provide guidance for the IESG in implementing reviews for each stage. RFC 2026 would have to be revised in order to put any change of this type into effect. That could be done by replacing RFC 2026 itself with a whole new document or by writing a short document that updates the standards track part of RFC 2026. 5.1 Alternate Standards Track Maturity Levels Internet specifications go through stages of development, testing, and acceptance. Within the Internet Standards Process, these stages are formally labeled "maturity levels". This section describes a set of alternate maturity levels and the expected characteristics of specifications at each level. 5.2 Stable Snapshot The entry-level maturity for the standards track is "Stable Snapshot". A specific action by the IESG is required to move a specification onto the standards track at the "Stable Snapshot" level. A Stable Snapshot specification is generally stable, has resolved known design choices, is believed to be well-understood, has received significant community review, and appears to enjoy enough community interest to be considered valuable. However, further experience might result in a change or even retraction of the specification before it advances. A Stable Snapshot should have no unknown technical omissions with respect to the requirements placed upon it. Any such omissions must be noted in the document. No such omission can endanger the security or stability of the Internet or of networks where the technology might be used. Implementers should treat Stable Snapshots as immature, pre-standard, specifications. It is desirable to implement them in order to gain experience and to validate, test, and clarify the specification. However, since the content of Stable Snapshots will be changed if problems are found or better solutions are identified, and will be changed as the technology is finalized, deploying implementations of such technologies into a disruption-sensitive environment is not recommended. Comments: This stage is designed to institutionalize and encourage the Bradner [Page 6] Internet-Draft Alternate Standards Track March 2004 current practice of vendors implementing from Internet Drafts while providing a way that a working group can indicate that they feel that a technology is stable enough to be so implemented and to provide a long-lived, unlike Internet Drafts, snapshot that the vendors can implement. Having vendors implement technology is an important quality check and meets the "running code" requirement of our motto. We want to encourage implementations whenever we can but this does need to be balanced with This is almost the same definition as RFC 2026 has for Proposed Standard. The major difference is that some of the technical requirements might not have yet been met. This is OK as long as any such holes in the specification are carefully noted in the document, except that there needs to be a complete enough security component so as to not endanger the networks where the technology is to be used, and that the technology not endanger the wellbeing of the network it will be run on. For example, a technology that requires reliable data transmission but is not compliant with RFC 2914 [RFC2914] would not be acceptable. The exact guidelines for the level of security required for a Stable Snapshot will evolve over time. In reviewing an Internet Draft for publication as a Stable Snapshot the IESG only needs to be sure that the working group has a reason to think that the technology is at a mature enough level that implementers can start to play with it and that the minimum security and 'health of the net' requirements have been met. The IESG should not try to ensure that the text is clear and unambiguous, the vendors will find that out while implementing and provide feedback to the working group. The IESG should not do a careful technology review as a precondition for publication as a Stable Snapshot. This process should be lightweight, not taking too much time on the part of the IESG or effort on the part of the working group and authors. The name, "Stable Snapshot" was chosen to clearly indicate that this is a pre-standard stage and to ensure that marketing people cannot easily misrepresent the status but there may be a better name that accomplishes the same goals. 5.3 Proposed Standard A Proposed Standard specification is generally stable, has resolved known design choices, is believed to be well-understood, has received significant community review, and appears to enjoy enough community interest to be considered valuable. Usually, neither implementation nor operational experience is required for the designation of a specification as a Proposed Bradner [Page 7] Internet-Draft Alternate Standards Track March 2004 Standard. However, such experience is highly desirable, and will usually represent a strong argument in favor of a Proposed Standard designation. Generally some documented level of implementation and/or operational experience is required prior to granting Proposed Standard status. However, the IESG may waive this requirement in order to allow a specification to advance to the Proposed Standard state when it is considered to be useful and necessary (and timely) even without any known implementations. A Proposed Standard should have no known technical omissions with respect to the requirements placed upon it. Implementers should treat Proposed Standards as stable, but perhaps not final, specifications. A Proposed Standard must be well- understood and known to be quite stable, both in its semantics and as a basis for developing an implementation. A Proposed Standard may still require additional or more widespread field experience, since it is possible for implementations based on Proposed Standard specifications to demonstrate unforeseen behavior when subjected to large-scale use in production environments. Comments: The requirements for publication as a Proposed Standard are mostly the same as currently in RFC 2026 for Proposed Standard with the addition of a requirement for at least some implementation experience. The IESG review for Proposed Standard could stay just like it is. The IESG should do the same careful technical review and a review to ensure that the language of the document is clear and precise as it has been doing for quite a while. Because most specifications for which publication as a Proposed Standard is requested will have been implemented I would expect that the IESG review will generally take less effort since the implementers experience will have uncovered unclear language and some or all technical issues, at least for the parts of the specification that had been implemented. There should be some documentation to show that there has been at least one implementation of a specification before the IESG authorizes the publication of the specification as a Proposed Standard. But the documentation does not need to be so detailed that it shows which individual options have been implemented. A list of the names of people or companies who have said they had implemented the specification should be sufficient. Bradner [Page 8] Internet-Draft Alternate Standards Track March 2004 Before adoption of a new description of Proposed Standard the IPR- related aspects should be revisited in list of the work in the IPR working group but I have not done that here. 5.4 Internet Standard A specification from which at least two independent and interoperable implementations from different code bases have been developed, and for which sufficient successful operational experience has been obtained, may be elevated to the "Internet Standard" level. For the purposes of this section, "interoperable" means to be functionally equivalent or interchangeable components of the system or process in which they are used. If patented or otherwise controlled technology is required for implementation, the separate implementations must also have resulted from separate exercise of the licensing process. Elevation to Internet Standard is a major advance in status, indicating a strong belief that the specification is mature and will be useful. The requirement for at least two independent and interoperable implementations applies to all of the options and features of the specification. In cases in which one or more options or features have not been demonstrated in at least two interoperable implementations, the specification may advance to the Internet Standard level only if those options or features are removed. The Working Group chair is responsible for documenting the specific implementations which qualify the specification for Draft or Internet Standard status along with documentation about testing of the interoperation of these implementations. The documentation must include information about the support of each of the individual options and features. This documentation should be submitted to the Area Director with the protocol action request. A Internet Standard (which may simply be referred to as a Standard) must be well-understood and known to be stable, both in its semantics and as a basis for developing an implementation. An Internet Standard is characterized by a high degree of technical maturity and by a generally held belief that the specified protocol or service provides significant benefit to the Internet community. An Internet Standard is considered to be a final specification, and changes should only be made to solve specific problems encountered. Comments: The description here is a combination of the descriptions of Draft Standard and Internet Standard in RFC 2026. One issue we have had over the years is just what does a working Bradner [Page 9] Internet-Draft Alternate Standards Track March 2004 group chair have to do to show multiple implementations of the base specification and all of the features. I have always felt that a simple spread sheet showing each feature, how many vendors claim to have the feature, and a checkbox to indicate that two or more vendors claim that they have tested implementations of the feature, would be just fine. But this turns out to be quite complex in some cases (see the Implementer's report for http 1.1 as an example). I am not sure if this turns out to be actually too much of an effort or just seems like too much of an effort. I still think it seems like about the right thing but the barrier to reach Internet Standard should be just as high as it needs to be but no higher. Since, in reality, there was little difference between the requirements in RFC 2026 for Draft Standard and Internet Standard, mostly a need to show market acceptance in some way, there seems to be no technical reason to preserve the different labels. 5.5 Minimum time in each stage. It seems to me that there needs to be a minimum time that a document must sit at a stage before it can move onward (as is the case in RFC 2026) just to be sure that problems are uncovered. I'm not sure if there is any way to figure out the ideal time so I would suggest that 6 months would be enough (as long as the rest of the requirements for the next level have been met). 6. Summary I've put out this proposal to stimulate discussion. There are a lot of details that would be needed to be worked out before actually proceeding but I do think that this would do the job of reestablishing the idea that it is worth the effort to move a document along the standards track while preserving the "running code" concept. 7. Security Considerations This document relates to IETF process, not any particular technology, thus it raises no particular security concerns. 8. Informative References [RFC2026] Bradner, S. Ed., "The Internet Standards Process -- Revision 3," October 1996, RFC 2026 [prob] Davies, E. Ed., "IETF Problem Statement," work in progress, July 2003 Bradner [Page 10] Internet-Draft Alternate Standards Track March 2004 [RFC2914] Floyd, S., "Congestion Control Principles," September 2000, RFC 2914 9. Authors Address Scott Bradner Harvard University 29 Oxford St. Cambridge MA, 02138 sob@harvard.edu +1 617 495 3864 10. Full copyright statement: Copyright (C) The Internet Society (2003). Except as set forth below, authors retain all their rights. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for rights in submissions defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/S HE REPRESENTS (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Bradner [Page 11]