IPSP Working Group W. Hardaker INTERNET-DRAFT Network Associates Laboratories draft-hardaker-eos-oops-02pre.txt November 2002 Object Oriented PDUs for SNMP draft-hardaker-eos-oops-02pre.txt Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Copyright Notice Copyright (C) The Internet Society (2002). All Rights Reserved. 1. Introduction This draft specifies some new PDU operations intended to optimize SNMP requests for performance. This draft is not a complete specification, but merely is intended to illustrate concepts which can then be discussed within the EoS working group. A complete specification will be completed after the agreed features and packet structure is fairly well solidified so the author doesn't have to rewrite large portions of stuff ;-). There are no error handling codes or elements of procedures defined yet. The concepts, however, have been thought out about a lot. Feedback on what you see within would be greatly appreciated at this point. Wes Hardaker [Page 1] INTERNET-DRAFT Object Oriented PDU's for SNMP Nov. 2002 Terminology in this draft is frequently inconsistent. Forgive the use of the old-school terminology (e.g., "agent") intermixed with new terminology (e.g., "command responder"). The other common error is the usage of the word "column" instead of "sub-identifier" or whatever the next draft for SMIv3 is going to call it (I forget and think it may be changing in the next draft anyway hence I didn't try to get it right in this document). This too will be fixed if a standardized specification on this work goes forward. Some of the stuff in this draft has yet to undergo debate. Particularly the Write operation is not solidified in my mind and will certainly be subject to debate as to whether they go to far into solving the problems that many people have pointed out with existing transactions. Please read the draft all the way through, including the section on "Design Considerations and Questions" before asking questions of the author or the working group. Finally, this is very much a working document. The descriptions described here-in may not even exactly match the author's thoughts, as the document hasn't been extensively proof-read yet. 1.1. Related work with the SMIng working group. In revamping this document and the contained PDUs, the direct correlations to the SMIng working group have since been dropped. They'll be reinstated shortly, but for now just know that all the PDUs contained within this document can be used with the new hierarchal structure of a newer SMIv3 MIB structure. To think about it, everywhere you see the word "BER" in a PDU definition, think "BER or recursive". IE, sub-elements within a SMIng working group will be encoded into a BER field in the same way that the containing request needed it. 1.2. Table of Contents 1. Introduction 1 1.1. Related work with the SMIng working group. 2 1.2. Table of Contents 2 2. Background and Motivations 3 2.1. GET-NEXT complexity 3 2.2. OID compression 4 2.3. Row operations 4 2.4. Complex object relationships 4 2.5. Retrieving of more data than is desired. 5 2.6. Row hole traversal and object grouping 5 2.7. Index parsing and OID length restrictions. 5 2.8. Transactions 6 Wes Hardaker [Page 2] INTERNET-DRAFT Object Oriented PDU's for SNMP Nov. 2002 2.9. Easy conversion to and from popular data formats 6 2.10. Human readable error strings 6 3. Transport Protocol Considerations 7 4. PDU definitions 7 4.1. ASN.1 definitions for the PDUs 7 4.2. New PDU component definitions 13 5. PDU processing 19 5.1. Processing a Get-Object-PDU 19 5.2. Write-Object-PDU transaction semantics 21 6. Examples 22 6.1. Retrieve a specific row from the ifTable 22 6.2. A multiple-packet example with a double filter. 23 7. Design Considerations and Questions 26 7.1. Return-field ranges 26 7.2. Write-Object-PDU complexity 27 7.3. Get-Object/Write-Object commonality. 27 7.4. Push Button Functionality. 28 8. Editor's Addresses: 28 9. Acknowledgements 28 10. References 28 10.1. Informative 28 11. Full Copyright Statement 29 2. Background and Motivations Many recognized problems exist within the SNMP protocol [OPREQ, IABNM]. Many of the known problems are addressed by the new protocol operations defined within this draft. The general problems trying to be solved are described in this section. 2.1. GET-NEXT complexity A large number of problems result from a devices inability to access it's data in a way that can be efficiently traversed when searching for the next object in a series of GET-NEXT or GET-BULK SNMP requests. Internally, many implementations differ and thus it would be impossible in many cases to define a MIB which would be efficient when implemented on every architecture. The operations defined in this document do not suffer from these problems, as the data can be returned in any order that the agent feels is appropriate. It MUST, however, return data in a consistent manner such that if the same data is requested twice the agent will return the data in the same order. The exact order in which objects are returned is implementation specific. It's the author's belief that every agent will benefit from this, Wes Hardaker [Page 3] INTERNET-DRAFT Object Oriented PDU's for SNMP Nov. 2002 and that most managers don't care in the first place. Manager's are likely to either need the data unsorted in the first place (e.g. may only care about the values each row and not the relationships between the rows) or the data will need to be sorted in a different fashion than the MIB indexing defines in the first place. If the manager does need the data sorted, it's more likely it'll have the power to more easily do sorting via utilization of database technologies that are not realistically available on many agent platforms. 2.2. OID compression SNMPv2 PDU consist of a sequence of varbinds and in many cases, when multiple objects are requested from the agent, the OIDs contained within have common prefixes. It is widely recognized as a waste of resources to duplicate this prefix repeatedly. The PDUs defined in this document only rely on a base OID (e.g., an object identifier pointing to a SNMP Table) once, and the sub- objects (e.g., a row in a SNMP Table) underneath it are defined as references from the base OID. Each index set which defines the proper reference for the remainder of the data components in a given "row" or "sub-object" is only included once as well. This allows large groups of data to be transmitted while only requiring a single OID to identify the top most grouping object (e.g., a table). 2.3. Row operations The ability to manipulate an entire row within a SNMP Table at once has long been a frustration of programmers implementing the protocol. The PDUs defined in this document are able to operate on entire groups of objects at once, where one of these groupings types is a row within a normal SNMP Table. 2.4. Complex object relationships Many people do not properly understand the interrelationships between SNMP Tables and have desired the ability to express their data in a more hierarchal fashion, such that SNMP tables can contain other SNMP tables. Although it is not possible to enable nested datatypes within SMIv2 today, the SMIng working group is trying to solve these problems and create the ability to define more complex relationships in SMIv3. The way object groupings are referenced within the PDUs defined in this document allows hierarchal object data to be transmitted Wes Hardaker [Page 4] INTERNET-DRAFT Object Oriented PDU's for SNMP Nov. 2002 efficiently and as a single group, thus allowing groups defined within SMIv3 to be efficiently transmitted within the PDUs defined in this document, new SMIv3-style objects be deployed. XXX: mention the intention to make the object definitions a bit extensible, so that new types requiring new parameters can be easily handled in the future without replacing the PDUs themselves. 2.5. Retrieving of more data than is desired. Extracting only the data needed from a SNMP Table has traditionally required pulling the entire table of data to the management station in many cases, when the indexes and their ordering was not sufficient for requesting subsets of the data desired. The request PDUs defined in this document allow more careful selection of indexing data and allow simple search criteria to be fulfilled. A delicate balance is required to ensure that the devices being polled for data are not taxed with expensive search requests, so the criteria imposed within these documents is limited to a limited set of operators which may or may not be supported by the command responder implementation. The intention is not to shift all the complex decision processing burden from the management station to the command responder, but to increase efficiency on the wire where possible. XXX: the agent will actually have the ability to gain improvements from this optimization since the manager will perform fewer requests as a whole. 2.6. Row hole traversal and object grouping When requests to GET, GET-NEXT, and GET-BULK data come back in a SNMP RESPONSE PDU, the data can be hard to organize back into a logical structure again. Holes of data (a row that contains no data for a particular column) within SNMP Tables make processing of the data collected more tedious. Additionally, GET-BULK responses interleave it's data, possibly with END-OF-MIB exceptions, which only adds further complexity to the data processing. Since the PDUs defined in this document deal directly with MIB objects as a whole, the data relationships within an object is preserved. This makes transferring large amounts of data efficient on both the sending and receiving side, as neither side needs to sort VarBinds properly. The data groupings are now appropriately marked within the packet itself. 2.7. Index parsing and OID length restrictions. Wes Hardaker [Page 5] INTERNET-DRAFT Object Oriented PDU's for SNMP Nov. 2002 Although the encoding and decoding of Table indexes into and out out OIDs is algorithmic, many SNMP software packages and toolkits make mistakes in implementing the algorithm. Additionally, since the maximum length of an OID is limited to 128 sub-identifiers, it limits the size of the index data that can be contained within an OID. This limitation results in poorly designed and broken MIBs and MIBs with less-than-ideal table indexes. The indexes within the PDUs defined in this document are encoded directly into the packet as opposed to being encoded into the OID first (which is then encoded into the packet). This simplifies both management and agent code and reduces the chances of one side incorrectly encoding or decoding the indexes. Additionally, it provides the ability to use a set of indexes which is larger than the OID-encoded imposed length of 255 sub-identifies. 2.8. Transactions The limited set of SNMP transactions have been difficult to cope with when large sets of data must be pushed around, since all the objects within a SNMP SET PDU must be set as if simultaneously. In actuality, configuration data often contains independent segments of data. Typically, independent sets of data must be pushed inside individual PDUS when configuring an agent, which is inefficient. The PDUs defined in this document define a less restrictive and more flexible transaction model that lets large quantities of data be pushed efficiently across the network and will still allow individual groupings to specify whether they are inter-related and must be treated as a single transaction or not and whether or not ordering is important 2.9. Easy conversion to and from popular data formats A desire has been shown by network operators for SNMP objects to be easily accessible and convertable to and from more human friendly expression languages and storage systems, like XML or SQL Databases. The objects within the PDUs contained in this document are designed for such a purpose. It should be, for example, trivial to convert them to and from a hierarchial XML schema set describing the objects in question. For the example XML case, the BER hierarchal packet format can be quickly converted to XML by replacing the sequence tags with XML ascii delimiter tags. 2.10. Human readable error strings Integer error codes are extremely useful for machine parsibility and interoperability, but it's frequently nice to have an extra error Wes Hardaker [Page 6] INTERNET-DRAFT Object Oriented PDU's for SNMP Nov. 2002 string that may be passed to an operator to assist in extra debugging of problems. The response PDUs defined in this document contain an "error string" for exactly this purpose. 3. Transport Protocol Considerations The PDUs defined in this document allow the transmission of large data sets in a more compressed form than previous SNMP PDUs, it is still recommended that large requests and responses be transmitted over a SNMP transmission domain which provides for guaranteed network packet delivery (e.g., TCP). Large responses, containing many objects, carried over transmission domains (e.g., UDP) which can not guarantee delivery are still likely to be problematic. It is well beyond the scope of this document to re-implement reliable transmission mechanisms. 4. PDU definitions This section describes the new PDUs defined by this document, both as ASN.1 definitions as well as textual descriptions of the pieces that make up the PDUs. The write support PDUs are less well defined and not entirely complete, but still worth reading. Note well: the SMIv3 extensions are not complete in this document. Some last minute rethinking caused them to be obsolete, and the definitions could not be updated in time for publishing of this draft. The SMIv2 applications should be fairly solid. 4.1. ASN.1 definitions for the PDUs SNMP-OOPS DEFINITIONS ::= BEGIN IMPORTS Integer32, ObjectSyntax, BITS FROM SNMPv2-SMI; -- -- Notes: ignore the OptionField for now. They'll be fixed later. -- (for instance, they should all be unique definitions) -- -- -- Get-Object-PDU definition -- Wes Hardaker [Page 7] INTERNET-DRAFT Object Oriented PDU's for SNMP Nov. 2002 Get-Object-PDU ::= [XXX] -- 9? SEQUENCE { request-id Integer32, option-field OptionField, request-objects RequestObjects } OptionField ::= SEQUENCE { } -- XXX needs formal definition RequestObjects ::= SEQUENCE (SIZE (0..max-bindings)) OF RequestObject RequestObject ::= SEQUENCE { max-return-objects Integer32, skip-objects Integer32, max-depth Integer32, cursor OCTET STRING, request-flags RequestCharacteristics, option-field OptionField, request-base-OID OBJECT IDENTIFIER, request-element-list ElementsRequestList, search-criteria SearchCriteria } RequestCharacteristics ::= IMPLICIT BIT STRING { restartOnInvalidCursor(0), returnAllDataOnSearchFailure(1) } ElementsRequestList ::= SEQUENCE (SIZE(0..max-bindings)) OF ElementSpecifier ElementSpecifier ::= CHOICE { index-number[0] -- apply criteria to an index IMPLICIT INTEGER, subcomponent-number[1] -- apply criteria to a subcomponet IMPLICIT INTEGER, -- for SMIv3 replace a 0.0 prefix with the baseOID Wes Hardaker [Page 8] INTERNET-DRAFT Object Oriented PDU's for SNMP Nov. 2002 multiple-subcomponent[2] -- apply to a sub[sub,...]component IMPLICIT OBJECT IDENTIFIER } SearchCriteria ::= SEQUENCE { match-type MatchType data CHOICE { -- sub-sequences for logical operations sub-criteria[0] IMPLICIT SEQUENCE (SIZE(0..max-bindings)) OF SearchCriteria -- match operation data match-data[1] IMPLICIT SEQUENCE { which ElementSpecifier what ObjectSyntax } } } MatchType ::= INTEGER { -- any datatypes: equals(0), -- note: equals NULL ::= exists not-equals(1), -- numerical only: lessThan(10), lessThanOrEqual(11), greaterThan(12), greaterThanOrEqual(13), -- binary comparisons: regexp(20), not-regexp(21), -- logical operations: logicalAND(100), logicalOR(101), logicalNOT(102) -- sub-criteria must be equal 1 in length if used -- ... IANA assigned up to 255 -- enterprise specific: -- 256*EnterpriseID to -- 256*EnterpriseID + 255 } Wes Hardaker [Page 9] INTERNET-DRAFT Object Oriented PDU's for SNMP Nov. 2002 -- -- Get-Object-Repsonse-PDU definition -- Get-Object-Repsonse-PDU ::= [XXX] -- 10? SEQUENCE { request-id Integer32, option-field OptionField, return-objects ReturnObjects } ReturnObjects ::= SEQUENCE (SIZE(0..max-bindings)) OF ReturnObject ReturnObject ::= SEQUENCE { error-information-list ErrorInformationList cursor OCTET STRING, response-flags RequestCharacteristics, option-field OptionField, request-base-OID OBJECT IDENTIFIER, returned-data-list DataList } -- should be an empty sequence if no errors occurred ErrorInformationList ::= SEQUENCE (SIZE(0..max-bindings)) OF ErrorInformation ErrorInformation ::= SEQUENCE { error-status ErrorCode, error-index OBJECT IDENTIFIER, error-string OCTET STRING } ErrorCode ::= INTEGER { noError(0), -- ... XXX: enumerations to be defined later } DataList ::= Wes Hardaker [Page 10] INTERNET-DRAFT Object Oriented PDU's for SNMP Nov. 2002 SEQUENCE (SIZE(0..max-bindings)) OF ObjectData ObjectData ::= SEQUENCE (SIZE(0..max-bindings)) OF ElementData ElementData ::= SEQUNECE { which-element ElementSpecifier, element-value ElementValue } ElementValue ::= CHOICE { base-type-object ObjectSyntax, -- for SMIv3 sub-objects: sub-object ReturnedDataList } -- -- Write-Object-PDU definition -- Write-Object-PDU ::= [XXX] -- 11? SEQUENCE { request-id Integer32, write-flags BIT STRING { returnDataOnlyOnError(0) }, option-field OptionField, write-transactions WriteTransactions } WriteTransactions ::= SEQUENCE (SIZE(0..max-bindings)) OF WriteTransaction WriteTransaction ::= SEQUENCE { transaction-flags TransactionSemantics option-field OptionField transaction-data-list TransactionDataList } TransactionSemantics ::= Wes Hardaker [Page 11] INTERNET-DRAFT Object Oriented PDU's for SNMP Nov. 2002 BIT STRING { needSuccess(0), needAll(1), notOrderDependent(2) } TransactionDataList ::= SEQUENCE (SIZE(0..max-bindings)) OF TransactionData TransactionData ::= CHOICE { create-transaction[0] CreateTransaction, modify-transaction[1] ModifyTransaction, delete-transaction[2] DeleteTransaction } CreateTransaction ::= { request-base-OID OBJECT IDENTIFIER, create-data-list DataList } ModifyTransaction ::= { request-base-OID OBJECT IDENTIFIER, modify-search-criteria SearchCriteria modify-data-list DataList } DeleteTransaction ::= { request-base-OID OBJECT IDENTIFIER, delete-search-criteria SearchCriteria delete-data-list DataList } -- -- Write-Object--Response-PDU definition -- Write-Object-Response-PDU ::= [XXX] -- 12? SEQUENCE { request-id Integer32, write-flags BIT STRING { returnDataOnlyOnError(0) }, option-field OptionField, error-information-list ErrorInformationList write-transaction-responses WriteTransactionResponses Wes Hardaker [Page 12] INTERNET-DRAFT Object Oriented PDU's for SNMP Nov. 2002 } WriteTransactionResponses ::= SEQUENCE (SIZE(0..max-bindings)) OF WriteTransactionResponse WriteTransactionResponse ::= SEQUENCE { transaction-flags TransactionSemantics option-field OptionField transaction-response-list TransactionResponseList } TransactionResponseList ::= SEQUENCE (SIZE(0..max-bindings)) OF TransactionResponse TransactionResponse ::= CHOICE { create-transaction[0] CreateResponse, modify-transaction[1] ModifyResponse, delete-transaction[2] DeleteResponse } CreateResponse ::= { request-base-OID OBJECT IDENTIFIER, create-data-list DataList -- copy of sent } ModifyResponse ::= { request-base-OID OBJECT IDENTIFIER, modify-data-list DataList -- ONLY modified elements } DeleteResponse ::= { request-base-OID OBJECT IDENTIFIER, delete-data-list DataList -- ONLY indexes } END 4.2. New PDU component definitions This section defines the pieces that make up new PDUs in human readable terms. The following components defined in the PDUs above are roughly Wes Hardaker [Page 13] INTERNET-DRAFT Object Oriented PDU's for SNMP Nov. 2002 described below. See the "PDU Processing" section for information on how processing of all of these elements together will happen. request-id The standard request-id concept that exists in previous PDUs defined by the previous versions of SNMP max-return-objects The maximum number of objects to return in one response mes­ sage. A value of 0 indicates "all". skip-objects Don't return the first skip-objects objects, but rather return objects numebering from (skip-objects + 1) to (max-return- objects + skip-objects + 1). A value of 0 indicates start returning with the first object. This value is entirely ignored if the cursor (see below) is valid. max-depth The maximum depth of a SMIv3 structure to return. This will be explained in greater detail in a later draft. cursor An OCTET STRING defining the starting point for collecting data. This MUST be a zero length string for the first request, and SHOULD contain the value in the response for chained requests which are expected to return future data. The value of the cursor is implementation dependent at the command responder. The manager MUST treat it as generic data and should use it when possible to obtain the next set of information in a series of requests. Command responders MUST refer to the value of the skip-objects field if the cursor field is a zero length string. Command responders MUST ignore the value of the skip-objects field if the cursor field is a non-zero length string and is deemed to be a valid cursor. Cursors SHOULD be defined by the command responder in such a way as to avoid referencing an existing object in such a way that if the existing object disappears the cursor will become invalid. I.E., CURSORS should be data independent where pos­ sible and should be a reference into the appropriate place in the storage mechanism and not a reference to a data piece itself within the storage mechanism. Wes Hardaker [Page 14] INTERNET-DRAFT Object Oriented PDU's for SNMP Nov. 2002 If a cursor is deemed invalid by the command responder and it is impossible for it to determine where to start collecting data from, the command responder must do one of two things: a) if the restartOnInvalidCursor flag is set, the search oper­ ation should be restarted from the beginning of all possible data. b) if the restartOnInvalidCursor flag is not set, the search operation should start at the point indicated by the skip- objects field. As an example cursor, an agent which was merely performing internal Get-Object-PDU translations to a GETNEXT translations might return a cursor containing the BER encoding of the last OID returned within the response. The next request could merely continue processing using the encoded OID just as if a real GETNEXT had come in. request-flags A flag list indicating particular features to be used when processing the PDU. option-field A sequence reserved for future use. Command responders which encounter unknown option types specified in this sequence MUST ignore the data and proceed as if the field itself didn't exist in the request. Future protocol extensions MUST define extensions to be implemented within this sequence in such a way that the option can be safely ignored by implementations which fail to understand it. This field exists in multiple places in the PDU definitions contained within this document. In all cases, the sequences MUST contain only an ASN.1 OPTION-ized list of parameters to be defined in future documents. The option index numbers for the sequence will be assigned and administered by IANA. Note: Proper ASN.1 definitions of the option fields in this document are missing. request-base-OID The base OID for a given element to request information from. For SMIv2 objects, this is either the OID pointing to a table or a particular scalar. For SMIv3 objects, it should point to Wes Hardaker [Page 15] INTERNET-DRAFT Object Oriented PDU's for SNMP Nov. 2002 the highest instantiation of an element. request-elements-list This sequence contains a list of subcomponents to be returned in the response. IE, what particular data elements should be returned. If empty, then all data for the given object referred to by the request-based-OID MUST be returned. Man­ agers SHOULD specify the data they wish to have returned rather than leave these fields empty. Managers SHOULD request accessible index elements be returned as component values rather indexes values if the indexes are accessible. search-criteria This sequence contains criteria which indicates which elements should be returned. If the object data to be returned does not match the criteria imposed by these restrictions in the request, then it shouldn't be returned. Get-Object-PDU implementations SHOULD specify column matching in preference to index matching, and thus index matching SHOULD ONLY be used for external index values which don't have a column number assignment within the object. match-type Specifies the criteria to be imposed for a given search value. match-type's are intended for simple expressions to help reduce the data sent over the wire. If the returnAllDataOnSearchFailure bit is set, then if a search operation fails due to a condition like an unsupported match-type, all data will be returned as if the search-type operator had always returned "true" (i.e., the Get-Object- Response should contain all possible data that the match-type would have been used to discriminate against). If the retur­ nAllDataOnSearchFailure bit is not set the related return- objects field will contain no data when sent back to the requester. In either case, an unSupportedSearchOperation error condition will be added to the error-information-list field of the Get-Object-Response-PDU and the error-index field will be set to the search component that caused the failure. Note that if the returnAllDataOnSearchFailure is set, data is still returned and thus the error can be viewed as a warning to the requester that more data was returned than requested. Implementations SHOULD return a returnAllDataOnSearchFailure Wes Hardaker [Page 16] INTERNET-DRAFT Object Oriented PDU's for SNMP Nov. 2002 error in the error-information-list for each unsupported unique match-type within the requested search-criteria. error-status An error status for the entire request operation, if an error occurred which affected the entire operation. Agent's SHOULD strive to put exceptions in the data field where possible instead. error-index An index into "something", where "something" is defined by what the error-status field contains (as an example, the unSupportedSearchRange error indicates it's an index into the search objects in the original request). It is an OBJECT IDENTIFIER which should begin with the 0.0 (zeroDotZero) pre­ fix, and the components of the identifier after that will indicate which sub-element of either the index-request-list, column-request-list or search-criteria field caused the prob­ lem. error-string A string describing the particular error. This SHOULD be set to a human readable string describing the problem in greater detail than the error-status field alone can indicate. Soft­ ware implementations MUST NOT expect the contents of field to be machine parsable. response-flags The flags field MUST be set according to how the response was handled within the responder. Ideally this should be an exact copy of the flags field from the request, assuming the flags were understood. Non-understood flags from the request MUST be set to 0 in the Get-Object-Response-PDU. write-flags Global flags to apply to the entire PDU. If the returnDataOn­ lyOnError bit is set, then the Write-Object-Response-PDU will only contain a duplicate set-objects portion of the message if there was an error somewhere in the processing of the message, otherwise the objects will be truncated. Command Responder implementations SHOULD support this flag, but if for any rea­ son they decide to return the set-objects data portion of the message they MUST NOT set the returnDataOnlyOnError bit. If Wes Hardaker [Page 17] INTERNET-DRAFT Object Oriented PDU's for SNMP Nov. 2002 the flag is set, and the command responder supports it, then the write-transaction-responses field must be an zero length sequence. transaction-flags Flags which apply to each sub-element of the transaction to be performed. Currently there are 3 bits to define transactional processing semantics to be used when processing this message. These values are described in greater detail in the next sec­ tion. TransactionData Indicates what type of operation is to be performed by this transaction set. Specifically: create: creates a new row. It is to be considered an error condition if either the row can not be created or if the row already existed. modify: modifies an existing row to replace some or all of the writable parameters within the row. It is an error condition if the row to be modified does not exist. delete: deletes a given row. If the row does not exist or can not be deleted for some reason, it is considered to be an error condition. The column-data-list MUST be empty when transmitting this operation. modify-search-criteria delete-search-criteria The search criteria imposed by the index-search-objects and column-search-objects are identical in operation to the Get- Object-PDU equivalents, except that instead of requested information back, the column-data field objects should be applied in the case of a create or a modify, or the rows should simply be deleted in the case of a delete. Unlike the Get-Object-PDU match-type handling, an unsupported match-type will always trigger an error condition and no modification or deletion will occur for that transaction (or any other trans­ actions dependent on the success of the one containing the error). Wes Hardaker [Page 18] INTERNET-DRAFT Object Oriented PDU's for SNMP Nov. 2002 5. PDU processing 5.1. Processing a Get-Object-PDU The following procedures describe processing for generating a Get- Object-Response-PDU. The Get-Object-Response-PDU MUST contain the same request-id as the Get-Object-PDU and MUST contain the same num­ ber of return-objects as there were request-objects in the Request- Object-PDU being processed and MUST appear in the same order. Apply the following procedure to each of the request-objects within the Get-Object-PDU to generate a corresponding return-objects sequence to be placed in the Get-Object-Response-PDU. Each request-object MUST generate a corresponding return-object with less than or equal to max-return-objects pieces as specified in the request-object, unless max-return-objects is zero. If max-return- objects is zero then all objects meeting the search-criteria MUST be returned. [XXX: transport issues] If the cursor field is specified in the request-object, the cursor field MUST be used, if possible, to determine a starting point for the data to be returned. If the contents of the cursor field are invalid or it is impossible to determine a valid start place and the restartOnInvalidCursor bit is set in the search-criteria, then the data should be searched from the beginning after skipping skip- objects number of objects. Note that when counting objects to skip, only objects which meet the requirements imposed by the search-cri­ teria WILL be counted. If the restartOnInvalidCursor bit is not set, then no objects should be returned in the return-objects list for this request. Regardless of whether the restartOnInvalidCursor bit is set or not, a invalidCursor error MUST be inserted into the error-information-list in the return-objects response. The cursor field MUST be a zero-length string in the return-object if max-return-objects is not 0 and there are less than max-return- objects being returned in the returned-data-list field. If the num­ ber of objects being returned is equal to the max-return-objects parameter, then the cursor field SHOULD be a zero length string if the last object being returned is the last object in the possible set of data for the given search-criteria. Implementations MUST return data in a dependable order. Successive requests with identical search-criteria and request-base-OIDs MUST return objects in the same order in which they were returned previ­ ously (i.e., if object A is returned before object B at one point in time, it should always be returned before object B (even if object C Wes Hardaker [Page 19] INTERNET-DRAFT Object Oriented PDU's for SNMP Nov. 2002 is returned in between). [XXX: ugh. needs better wording] [XXX: describe max-depth if it stays around] When the maximum number of objects to be returned have reached, as specified by the max-return-objects portion of the request, a cursor MUST be constructed which meets the criteria described earlier in the document [see XXX]. It MUST be usable in the future to deter­ mine the starting place of a follow on request. It SHOULD [XXX: maybe this should be a MUST?] be usable even if the portions of data near where the cursor marks is modified or deleted. When searching for objects which are to be returned to the requester, the search-criteria MUST be consulted. Any object not matching the supported elements of the criteria imposed by the search-criteria field MUST NOT be included in the response. When processing the search-criteria field, any match-type which is not supported by the processing engine MUST result in a unsupported­ MatchType error to be inserted into the error-information list. Additionally, depending on the value of the returnAllDataOnSearch­ Failure flag in the request-characteristics field: a) if the returnAllDataOnSearchFailure bit is set, then the evaluation MUST return all the data as if the unsup­ ported match-type imposed no filtering on the data. Gen­ erally, this would mean that any match-type should be considered as evaluating to true, except in the case where the evaluation is enclosed directly within a logi­ calNOT operation, in which case the logicalNOT must eval­ uate to true. b) if the returnAllDataOnSearchFailure bit is unset, then the response MUST contain an empty returned-data-list. Each unsupported match-type contained in the search-criteria field MUST generate a unique unsupportedMatchType error in error-informa­ tion for the given request. Logical operations WILL use the sub-criteria field of the SearchCri­ teria to specify a list of sub-criteria the logical operation is to operate on. If the match type is logicalNOT there MUST be exactly one element in the sub-criteria list. When constructing a return-objects list, the request-element-list should be consulted to determine which elements of a given object are to be returned in the response. When sending a Get-Object-PDU, the request-element-list SHOULD use the subcomponent-number speci­ fier when possible in preference to the index-number specifier when Wes Hardaker [Page 20] INTERNET-DRAFT Object Oriented PDU's for SNMP Nov. 2002 the index is an accessible column. 5.2. Write-Object-PDU transaction semantics Note: this section is not entirely finished, and this section cur­ rently only describes the less-than-obvious semantics surround transactional handling. The flags defining Write-Object semantics can be summarized as fol­ lows: needSuccess If the needSuccess bit is turned on, it means that this transaction is to be considered a failed transaction if during any time an error condition occurs. If at anytime a failure occurs when processing the transaction-data- list, the entire transaction must be rolled back. needAll If needAll is specified, it means each sub-component in the transaction-data-list must be attempted. notOrderDependent If the notOrderDependent bit is turned on, the transac­ tions contained in the transaction-data-list MAY be exe­ cuted in an order, or in parallel. Note that they must still revert to their previous values in an error condi­ tion requires a state rollback. Thus, the following combinations of the first two flags can be bet­ ter described in combination: needSuccess = true, needAll = true: doAll [doUntilFailure] In this case, all contained transactions must succeed. If any failure occurs, the entire set must be rolled back. needSuccess = false, needAll = true: tryAll With this combination set, the command responder must try to achieve all the transactions, however if any fail it is not required to roll back the rest of the transac­ tions. The failed transactions themselves, however, must individually revert to the before-write state. needSuccess = true, needAll = false: doAtLeastOne [doUntil­ Success] In this case, at least one must succeed for the transac­ tion as a whole to succeed. More importantly, if any one Wes Hardaker [Page 21] INTERNET-DRAFT Object Oriented PDU's for SNMP Nov. 2002 transaction component does succeed, processing MUST be stopped and the transaction as a whole is considered suc­ cessful. Another way to put it: at most one successful component is executed never more than one. needSuccess = false, needAll = false: tryAtLeastOne In this case, at least one must succeed for the transac­ tion as a whole to succeed. More importantly, if any one transaction component does succeed, processing MUST be stopped and the transaction as a whole is considered suc­ cessful. Another way to put it: at most one successful component is executed never more than one. However, in this case even if none of the transaction components suc­ ceeds, the transaction as a whole is still considered successful. It is impossible for a transaction of this type to be considered a failure in itself. Also, error reporting will always occur on failed objects even if they don't affect the containing transactions and surrounding trans­ actions. Note that more data can be returned in many cases when search opera­ tions required operations on multiple rows based on the search cri­ teria. 6. Examples Here are some example requests and responses of data retrieval. In each case, the curly brases indicate a sequence of some kind within BER. 6.1. Retrieve a specific row from the ifTable This example retrieves the ifDescr and ifType columns for the 5th interface from the ifTable The Request: Get-Object-PDU { request-id 1 option-field { } request-objects { { max-return-objects 0 -- get all skip-objects 0 -- starting with the first max-depth 0 -- TBD cursor "" -- 0 length for first request flags "" -- none specified. option-field {} Wes Hardaker [Page 22] INTERNET-DRAFT Object Oriented PDU's for SNMP Nov. 2002 request-base-object IF-MIB::ifTable request-element-list { subcomponent-number = 2, subcomponent-number = 3 } search-criteria { match-type = 0, -- equals match-data = { which = 1, -- ifIndex what = 5 -- 5th } } } } } The Response: Get-Object-Response-PDU { request-id 1 option-field {} response-objects ::= { { error-information-list {} -- no Errors cursor "" -- no other data in search-criteria response-flags "" -- none specified. option-field {} -- no options request-base-object IF-MIB::ifTable returned-data-list { { { which-element = 2, element-value = "interface 5" }, { which-element = 3, element-value = 6 } } } } } } 6.2. A multiple-packet example with a double filter. This example shows the retrieval of the elements of the ifTable which are of type "ethernetCsmacd" and have a ifSpeed >= 10,000,000 bps. Retrieve only 1 row at a time. Note that the response has the objects returned in the order the remote agent specified [interface 12, then interface 5], not in something that would directly corre­ sponds to the older-style lexicographical ordering. The order returned must be consistent from one request to the next (ie, inter­ face 12 will exist before interface 5 in any request at any point in time where they are both to be returned), but the manner in which the ordering is accomplished is implementation specific. The cursor Wes Hardaker [Page 23] INTERNET-DRAFT Object Oriented PDU's for SNMP Nov. 2002 is used to indicate to the base agent where the request should restart from. The first request: Get-Object-PDU { request-id 2 option-field { } request-objects { { max-return-objects 1 -- only return 1 skip-objects 0 -- starting with the first max-depth 0 -- TBD cursor "" -- 0 length for first request flags "" option-field {} request-base-object IF-MIB::ifTable -- get ifIndex, ifDescr, and ifType request-element-list { subcomponent-number = 1, subcomponent-number = 2, subcomponent-number = 3 } search-criteria { match-type = 100, -- AND sub-criteria = { { match-type = 0, -- equals match-data = { which = 3, -- ifType what = 6 -- ethernetCsmacd } }, { match-type = 12, -- greaterThanOrEqual match-data = { which = 5, -- ifSpeed what = 10000000 } } } } } } } The first response: Get-Object-Response-PDU { request-id 2 option-field {} response-objects ::= { { Wes Hardaker [Page 24] INTERNET-DRAFT Object Oriented PDU's for SNMP Nov. 2002 error-information-list {} -- no Errors cursor "interface 12" -- agent implementation cursor. response-flags "" option-field {} request-base-object IF-MIB::ifTable returned-data-list { { { which-element = 1, element-value = 12 }, { which-element = 2, element-value = "interface 12" }, { which-element = 3, element-value = 6 } } } } } } The second request to obtain the rest of the data: Get-Object-PDU { request-id 3 option-field { } request-objects { { max-return-objects 1 -- only return 1 skip-objects 0 -- functionally not used max-depth 0 -- TBD cursor "interface 12" -- continue flags "" option-field {} request-base-object IF-MIB::ifTable -- get ifIndex, ifDescr, and ifType request-element-list { subcomponent-number = 1, subcomponent-number = 2, subcomponent-number = 3 } search-criteria { match-type = 100, -- AND sub-criteria = { { match-type = 0, -- equals match-data = { which = 3, -- ifType what = 6 -- ethernetCsmacd } }, Wes Hardaker [Page 25] INTERNET-DRAFT Object Oriented PDU's for SNMP Nov. 2002 { match-type = 12, -- greaterThanOrEqual match-data = { which = 5, -- ifSpeed what = 10000000 } } } } } } } The second response: Get-Object-Response-PDU { request-id 3 option-field {} response-objects ::= { { error-information-list {} -- no Errors cursor "" -- no cursor == last obj response-flags "" option-field {} request-base-object IF-MIB::ifTable returned-data-list { { { which-element = 1, element-value = 5 }, { which-element = 2, element-value = "interface 5" }, { which-element = 3, element-value = 6 } } } } } } 7. Design Considerations and Questions [this section is slightly out of date, but still worth reading] A lot of thought has gone into the PDUs defined in this document. I've tried to achieve a balance between flexibility and interoper­ ability that will lean toward interoperability most of the time. Comments welcome. This section describes some of the things that I'm still thinking about and would like feedback on. 7.1. Return-field ranges Wes Hardaker [Page 26] INTERNET-DRAFT Object Oriented PDU's for SNMP Nov. 2002 Currently the list of columns/indexes to return contains only a list of integers, which could be long if you needed every column from 1-25 but not 26-30, for example. Should we support the notion of "1-25" and if we do, should we only support ranges or should we try and mix and match. IE, how do you expresss "1-10 and 15 and 20-30": a) 1,10,15,15,20,30 b) {1,10},15,{20,30} {}s would have to specially tagged sequences c) 1,2,3,...,10,15,20,21,...,30 I'm leaning toward option (c) 7.2. Write-Object-PDU complexity In the long run, I made the Write-Object-PDU more powerful than I was originally thinking. It took me a while to decide what to do, but the more I thought about it the processing of the requests con­ tained within will not be that difficult. Specifically, the opera­ tions are very recursive in nature and the complexity when dealing with one specific component is isolated. Rollback to some extent is still going to be necessary, but the PDU likely reduces the amount of UNDO processing due to the fact that you can encode failure cases which may be known to be problematic and can be ignored. I debated for a while whether to put create/delete/modify transac­ tions with the same request, but I've come to believe that it's very necessary. Specifically, it's likely that you'll only want to delete a row if and only if the creation of another row succeeds. If either of these fail, then the entire transaction should fail else the box may be left in an un-usable state. However, there are also likely times that they are completely unrelated transactions, and thus can be flagged as independent using the needSuccess bit. One possible simplification would be to remove the recursiveness of the transaction and to leave everything else. If I were to remove anything, this would likely be it. I don't actually think it'll remove processing code within the agent, but it may lead to greater interoperability. I'm not at all convinced that's the case, how­ ever. 7.3. Get-Object/Write-Object commonality. The more I think about it, the more that I'm beginning to lean toward the idea that the Get-Object and Write-Object-PDUs should be similar in design. I debated for a long time whether the search criteria should go into the Write-Object-PDU or not, and I'm still not entirely decided (but it's there now). Wes Hardaker [Page 27] INTERNET-DRAFT Object Oriented PDU's for SNMP Nov. 2002 The danger comes in non-equal sign search criteria where you might think you know all the data on the far side that you're going to operate on, but it turns out that new rows have added by another source and thus you might affect a row you didn't otherwise intend to. However, I'm currently leaning toward the thought that the power to affect multiple objects with one transaction would be greatly beneficial. It will certainly help scaling with respect to management of extremely large networks containing lots of data which needs to be modified. In either case, it would be highly beneficial to retrieve a list of all the rows which were modified, thus the Write-Object transactions are designed such that the indexes of all the datasets which have been affected by the transaction are returned to the caller. This is one way to check for errors in the case of external modifica­ tions. So, the real question is: Do we want search criteria in Write-Object operations? I'm still leaning toward yes (even more so after writ­ ing this section). 7.4. Push Button Functionality. SNMP usage has proven that method invocation is needed within the protocol. The SET semantics and Write-Object semantics do not make this easy. Should a "invocation" write method be put in here now, even though we don't have a SMI that can define it's usage yet? 8. Editor's Addresses: Wes Hardaker P.O. Box 382 Davis, CA 95617 Email: hardaker@tislabs.com 9. Acknowledgements Many people participated in various discussions about the PDUs defined in this document. Special thanks go to: David Perkins (Riverstone Networks and SNMPinfo) Randy Presuhn (BMC) Dave Shield (University of Liverpool) Steve Waldbusser (Next Beacon) Glenn Waters (Nortel Networks) 10. References 10.1. Informative Wes Hardaker [Page 28] INTERNET-DRAFT Object Oriented PDU's for SNMP Nov. 2002 [OPREQ] Woodcock, D., "Operator Requirements of Infrastructure Management Methods", Internet Draft draft-ops-operator-req-mgmt-02.txt, expired August 2002. [IABNM] Schoenwaelder, J., "Overview of the 2002 IAB Network Management Workshop", Internet Draft draft-iab-nm-workshop-00.txt, expires April 2003. 11. Full Copyright Statement Copyright (C) The Internet Society (2002). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of develop­ ing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 12. Humor GOP GORP? WOP WORP. NOP NORP! Wes Hardaker [Page 29]