MPLS Working Group W. Cheng Internet-Draft China Mobile Intended status: Standards Track X. Min, Ed. Expires: 16 December 2023 ZTE Corp. T. Zhou Huawei J. Dai FiberHome Y. Peleg Broadcom 14 June 2023 Encapsulation For MPLS Performance Measurement with Alternate Marking Method draft-ietf-mpls-inband-pm-encapsulation-06 Abstract This document defines the encapsulation for MPLS performance measurement with alternate marking method, which performs flow-based packet loss, delay, and jitter measurements on MPLS live traffic. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on 16 December 2023. Copyright Notice Copyright (c) 2023 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/ license-info) in effect on the date of publication of this document. Cheng, et al. Expires 16 December 2023 [Page 1] Internet-Draft Encap for MPLS PM with AMM June 2023 Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 1.1. Conventions Used in This Document . . . . . . . . . . . . 3 1.1.1. Abbreviations . . . . . . . . . . . . . . . . . . . . 3 1.1.2. Requirements Language . . . . . . . . . . . . . . . . 4 2. Flow-based PM Encapsulation in MPLS . . . . . . . . . . . . . 4 2.1. Examples for Applying Flow-ID Label in a label stack . . 5 3. Procedures of Encapsulation, Look-up and Decapsulation . . . 8 4. Procedures of Flow-ID allocation . . . . . . . . . . . . . . 9 5. FLC and FRLD Considerations . . . . . . . . . . . . . . . . . 10 6. Equal-Cost Multipath Considerations . . . . . . . . . . . . . 11 7. Security Considerations . . . . . . . . . . . . . . . . . . . 11 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 11 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 12 10. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 12 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 12 11.1. Normative References . . . . . . . . . . . . . . . . . . 12 11.2. Informative References . . . . . . . . . . . . . . . . . 13 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 14 1. Introduction [RFC9341] describes a performance measurement method, which can be used to measure packet loss, delay, and jitter on live traffic. Since this method is based on marking consecutive batches of packets, it's referred to as Alternate-Marking Method. [RFC8372] describes the desired capabilities for MPLS flow identification, intended for in-band performance monitoring of MPLS flows. This document defines the encapsulation for MPLS performance measurement with alternate marking method, which performs flow-based packet loss, delay, and jitter measurements on MPLS live traffic. The encapsulation defined in this document supports performance monitoring at the intermediate nodes, as well as MPLS flow identification at both transport and service layers. Cheng, et al. Expires 16 December 2023 [Page 2] Internet-Draft Encap for MPLS PM with AMM June 2023 This document employs an encapsulation method, other than Synonymous Flow Label (SFL), to achieve MPLS flow identification. The method described in this document is complementary to the SFL method [RFC8957] [I-D.ietf-mpls-sfl-control], the former mainly aims at hop- by-hop processing and the latter mainly aims at edge-to-edge processing. Different sets of MPLS flows may use different methods. The method described in this document is also complementary to the In-situ OAM method [RFC9197] [RFC9326], the former doesn't introduce any new header whereas the latter introduces a new In-situ OAM header. Furthermore, the former requires the network nodes to collect the data used for performance measurement, while the latter requires the network nodes to collect the data used for operational and telemetry information collection. An MPLS flow may apply both of the two methods concurrently. 1.1. Conventions Used in This Document 1.1.1. Abbreviations ACL: Access Control List BoS: Bottom of Stack cSPL: Composite Special Purpose Label ECMP: Equal-Cost Multipath ELC: Entropy Label Capability ERLD: Entropy Readable Label Depth eSPL: Extended Special Purpose Label FL: Flow-ID Label FLC: Flow-ID Label Capability FLI: Flow-ID Label Indicator FRLD: Flow-ID Readable Label Depth LSP: Label Switched Path MPLS: Multi-Protocol Label Switching NMS: Network Management System Cheng, et al. Expires 16 December 2023 [Page 3] Internet-Draft Encap for MPLS PM with AMM June 2023 PHP: Penultimate Hop Popping PM: Performance Measurement PW: PseudoWire SFL: Synonymous Flow Label SID: Segment ID SR: Segment Routing TC: Traffic Class TTL: Time to Live VC: Virtual Channel VPN: Virtual Private Network XL: Extension Label 1.1.2. Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here. 2. Flow-based PM Encapsulation in MPLS Flow-based MPLS performance measurement encapsulation with alternate marking method has the following format: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Extension Label (15) | TC |S| TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Flow-ID Label Indicator (TBA1) | TC |S| TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Flow-ID Label |L|D|T|S| TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 1: Flow-based PM Encapsulation in MPLS Cheng, et al. Expires 16 December 2023 [Page 4] Internet-Draft Encap for MPLS PM with AMM June 2023 The Flow-ID Label Indicator (FLI) is an Extended Special Purpose Label (eSPL), which is combined with the Extension Label (XL, value 15) to form a Composite Special Purpose Label (cSPL), as defined in [RFC9017]. The FLI is defined in this document as value TBA1. The Traffic Class (TC) and Time To Live (TTL) [RFC3032] for the XL and FLI SHOULD follow the same field values of that label immediately preceding the XL. Otherwise, the TC and TTL for the XL and FLI MAY be different values if it is known that the XL will not be exposed as the top label at any point along the LSP. The Bottom of Stack (BoS) bit [RFC3032] for the XL and FLI MUST be zero. The Flow-ID Label (FL) is used as an MPLS flow identification [RFC8372], its value MUST be unique within the administrative domain. Flow-ID values can be allocated by an external NMS/controller, based on measurement object instance such as LSP or PW. There is a one-to- one mapping between Flow-ID and flow. The specific method on how to allocate the Flow-ID values is described in Section 4. The FL can be placed at either the bottom or the middle of the MPLS label stack, and the FL MAY appear multiple times in a label stack. Section 2.1 of this document provides several examples to illustrate how to apply FL in a label stack. The TTL for the FL MUST be zero to ensure that it is not used inadvertently for forwarding. The BoS bit for the FL depends on whether the FL is placed at the bottom of the MPLS label stack. Besides the flow identification, a color-marking field is also necessary for alternate marking method. To achieve the purpose of coloring the MPLS traffic, as well as the distinction between hop-by- hop measurement and edge-to-edge measurement, the TC for the FL is defined as follows: * L(oss) bit is used for coloring the MPLS packets for loss measurement. * D(elay) bit is used for coloring the MPLS packets for delay/jitter measurement. * T(ype) bit is used to indicate the measurement type. When T bit is set to 1, that means edge-to-edge performance measurement. When T bit is set to 0, that means hop-by-hop performance measurement. 2.1. Examples for Applying Flow-ID Label in a label stack Three examples on different layout of Flow-ID label (4 octets) are illustrated as follows: Cheng, et al. Expires 16 December 2023 [Page 5] Internet-Draft Encap for MPLS PM with AMM June 2023 (1) Layout of Flow-ID label when applied to MPLS transport. +----------------------+ | LSP | | Label | +----------------------+ <--+ | Extension | | | Label | | +----------------------+ |--- cSPL | Flow-ID Label | | | Indicator | | +----------------------+ <--+ | Flow-ID | | Label | +----------------------+ | Application | | Label | +----------------------+ <= Bottom of stack | | | Payload | | | +----------------------+ Figure 2: Applying Flow-ID to MPLS transport Note that here if the penultimate hop popping (PHP) is in use, the PHP LSR that recognizes the cSPL MAY choose not to pop the cSPL and the following Flow-ID label, otherwise the egress LSR would be excluded from the performance measurement. Also note that in other examples of applying Flow-ID to MPLS transport, one LSP label can be substituted by multiple SID labels in the case of using SR Policy, and the combination of cSPL and Flow-ID label can be placed between SID labels, as specified in Section 5. (2) Layout of Flow-ID label when applied to MPLS service. Cheng, et al. Expires 16 December 2023 [Page 6] Internet-Draft Encap for MPLS PM with AMM June 2023 +----------------------+ | LSP | | Label | +----------------------+ | Application | | Label | +----------------------+ <--+ | Extension | | | Label | | +----------------------+ |--- cSPL | Flow-ID Label | | | Indicator | | +----------------------+ <--+ | Flow-ID | | Label | +----------------------+ <= Bottom of stack | | | Payload | | | +----------------------+ Figure 3: Applying Flow-ID to MPLS service Note that here the application label can be MPLS PW label, MPLS Ethernet VPN label or MPLS IP VPN label, and it's also called VC label as defined in [RFC4026]. (3) Layout of Flow-ID label when applied to both MPLS transport and MPLS service. Cheng, et al. Expires 16 December 2023 [Page 7] Internet-Draft Encap for MPLS PM with AMM June 2023 +----------------------+ | LSP | | Label | +----------------------+ <--+ | Extension | | | Label | | +----------------------+ |--- cSPL | Flow-ID Label | | | Indicator | | +----------------------+ <--+ | Flow-ID | | Label | +----------------------+ | Application | | Label | +----------------------+ <--+ | Extension | | | Label | | +----------------------+ |--- cSPL | Flow-ID Label | | | Indicator | | +----------------------+ <--+ | Flow-ID | | Label | +----------------------+ <= Bottom of stack | | | Payload | | | +----------------------+ Figure 4: Applying Flow-ID to both MPLS transport and MPLS service Note that for this example the two Flow-ID values appearing in a label stack MUST be different, that is to say, the Flow-ID label applied to MPLS transport and the Flow-ID label applied to MPLS service share the same value space. Also note that the two Flow-ID label values are independent from each other, e.g., two packets can belong to the same VPN flow but two different LSP flows, or two packets can belong to two different VPN flows but the same LSP flow. 3. Procedures of Encapsulation, Look-up and Decapsulation The procedures for Flow-ID label encapsulation, look-up and decapsulation are summarized as follows: * The ingress node inserts the XL, FLI and FL into the MPLS label stack. At the same time, the ingress node sets the Flow-ID value, the two color-marking bits and the T bit, as defined in Section 2. Cheng, et al. Expires 16 December 2023 [Page 8] Internet-Draft Encap for MPLS PM with AMM June 2023 * If the hop-by-hop measurement is applied, i.e., the T bit is set to 0, then whether the transit node or the egress node is the processing node. If the edge-to-edge measurement is applied, i.e., the T bit is set to 1, then only the egress node is the processing node. The processing node looks up the FL with the help of the XL and FLI, and exports the collected data, such as the Flow-ID, block counters and timestamps, to an external NMS/ controller, referring to the alternate marking method. Note that while looking up the Flow-ID label, the transit node needs to perform some deep packet inspection beyond the label (at the top of the label stack) used to take forwarding decisions. * The processing node may also pop the XL, FLI and FL from the MPLS label stack. The egress node pops the whole MPLS label stack, and this document doesn't introduce any new process to the decapsulated packet. 4. Procedures of Flow-ID allocation There are two ways of allocating Flow-ID, one way is to allocate Flow-ID by manual trigger from the network operator, and the other way is to allocate Flow-ID by automatic trigger from the ingress node. Details are as follows: * In the case of manual trigger, the network operator would manually input the characteristics (e.g. IP five tuples and IP DSCP) of the measured flow, then the NMS/controller would generate one or two Flow-IDs based on the input from the network operator, and provision the ingress node with the characteristics of the measured flow and the corresponding allocated Flow-ID(s). * In the case of automatic trigger, the ingress node would identify the flow entering the measured path, export the characteristics of the identified flow to the NMS/controller by IPFIX [RFC7011], then the NMS/controller would generate one or two Flow-IDs based on the characteristics exported from the ingress node, and provision the ingress node with the characteristics of the identified flow and the corresponding allocated Flow-ID(s). Cheng, et al. Expires 16 December 2023 [Page 9] Internet-Draft Encap for MPLS PM with AMM June 2023 The policy pre-configured at the NMS/controller decides whether one Flow-ID or two Flow-IDs would be generated. If the performance measurement on MPLS service is enabled, then one Flow-ID applied to MPLS service would be generated; If the performance measurement on MPLS transport is enabled, then one Flow-ID applied to MPLS transport would be generated; If both of them are enabled, then two Flow-IDs respectively applied to MPLS service and MPLS transport would be generated, in this case, the transit node needs to look up both of the two Flow-IDs by default, and that can be changed by configuration to, e.g., look up only the Flow-ID applied to MPLS transport. Whether using manual trigger or automatic trigger, the NMS/controller MUST guarantee every generated Flow-ID is unique within the administrative domain and MUST NOT have a value in the reserved label space (0-15) [RFC3032]. 5. FLC and FRLD Considerations Analogous to the Entropy Label Capability (ELC) defined in Section 5 of [RFC6790] and the Entropy Readable Label Depth (ERLD) defined in Section 4 of [RFC8662], the Flow-ID Label Capability (FLC) and the Flow-ID Readable Label Depth (FRLD) are defined in this document. Both FLC and FRLD have the similar semantics with the ELC and ERLD to a router, except that the Flow-ID is used in its flow identification function while the Entropy is used in its load-balancing function. The ingress node MUST insert each FL at an appropriate depth, which ensures the node to which the FL is exposed has the FLC. The ingress node SHOULD insert each FL within an appropriate FRLD, which is the minimum FRLD of all the on-path nodes that need to read and use the FL in question. How the ingress node knows the FLC and FRLD of all the on-path nodes is outside the scope of this document, whereas [I-D.xzc-lsr-mpls-flc-frld] provides a method to achieve that. When the SR paths are used for transport, the label stack grows as the number of on-path segments increases, if the number of on-path segments is high, that may become a challenge for the FL to be placed within an appropriate FRLD. In order to overcome this potential challenge, an implementation MAY provide flexibility to the ingress node to place FL between SID labels, i.e., multiple identical FLs at different depths MAY be interleaved with SID labels, when that happens a sophisticated network planning may be needed and it's beyond the scope of this document. Cheng, et al. Expires 16 December 2023 [Page 10] Internet-Draft Encap for MPLS PM with AMM June 2023 6. Equal-Cost Multipath Considerations Analogous to what's described in Section 5 of [RFC8957], under conditions of Equal-Cost Multipath (ECMP), the introduction of the FL may lead to the same problem as caused by the SFL, and the two solutions proposed for SFL would also apply here. 7. Security Considerations This document introduces the performance measurement domain that is the scope of a Flow-ID label. The Flow-ID Label Indicator and Flow- ID label MUST NOT be signaled and distributed outside one performance measurement domain. Improper configuration so that the Flow-ID label being passed from one domain to another would likely result in potential Flow-ID conflicts. To prevent packets carrying Flow-ID label from leaking from one domain to another, the domain boundary nodes SHOULD deploy some policies (e.g., ACL) to filter out the packets. Specifically, in the sending edge, the domain boundary node SHOULD filter out the packets that carry the Flow-ID Label Indicator and are sent to other domain; in the receiving edge, the domain boundary node SHOULD drop the packets that carry the Flow-ID Label Indicator and are from other domains. 8. IANA Considerations In the Special-Purpose MPLS Label Values registry, a new Extended Special-Purpose MPLS Label Value for the Flow-ID Label Indicator is requested from IANA as follows: +==========================+===============+============+===========+ | Extended Special-Purpose | Description | Semantics | Reference | | MPLS Label Value | | Definition | | +==========================+===============+============+===========+ | TBA1 | Flow-ID | Section 2 | This | | | Label | | Document | | | Indicator | | | +--------------------------+---------------+------------+-----------+ Table 1: New Extended Special-Purpose MPLS Label Value for Flow- ID Label Indicator Cheng, et al. Expires 16 December 2023 [Page 11] Internet-Draft Encap for MPLS PM with AMM June 2023 9. Acknowledgements The authors would like to acknowledge Loa Andersson, Tarek Saad, Stewart Bryant, Rakesh Gandhi, Greg Mirsky, Aihua Liu, Shuangping Zhan, Ming Ke, Wei He, and Ximing Dong for their careful review and very helpful comments. The authors would like to acknowledge Italo Busi and Chandrasekar Ramachandran for their insightful MPLS-RT review and very helpful comments. 10. Contributors Minxue Wang China Mobile Email: wangminxue@chinamobile.com Wen Ye China Mobile Email: yewen@chinamobile.com 11. References 11.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . [RFC3032] Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y., Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack Encoding", RFC 3032, DOI 10.17487/RFC3032, January 2001, . [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, . [RFC9017] Andersson, L., Kompella, K., and A. Farrel, "Special- Purpose Label Terminology", RFC 9017, DOI 10.17487/RFC9017, April 2021, . [RFC9341] Fioccola, G., Ed., Cociglio, M., Mirsky, G., Mizrahi, T., and T. Zhou, "Alternate-Marking Method", RFC 9341, DOI 10.17487/RFC9341, December 2022, . Cheng, et al. Expires 16 December 2023 [Page 12] Internet-Draft Encap for MPLS PM with AMM June 2023 11.2. Informative References [I-D.ietf-mpls-sfl-control] Bryant, S., Swallow, G., and S. Sivabalan, "A Simple Control Protocol for MPLS SFLs", Work in Progress, Internet-Draft, draft-ietf-mpls-sfl-control-03, 6 August 2022, . [I-D.xzc-lsr-mpls-flc-frld] Min, X., Zhang, Z., and W. Cheng, "Signaling Flow-ID Label Capability and Flow-ID Readable Label Depth", Work in Progress, Internet-Draft, draft-xzc-lsr-mpls-flc-frld-02, 10 February 2023, . [RFC4026] Andersson, L. and T. Madsen, "Provider Provisioned Virtual Private Network (VPN) Terminology", RFC 4026, DOI 10.17487/RFC4026, March 2005, . [RFC6790] Kompella, K., Drake, J., Amante, S., Henderickx, W., and L. Yong, "The Use of Entropy Labels in MPLS Forwarding", RFC 6790, DOI 10.17487/RFC6790, November 2012, . [RFC7011] Claise, B., Ed., Trammell, B., Ed., and P. Aitken, "Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of Flow Information", STD 77, RFC 7011, DOI 10.17487/RFC7011, September 2013, . [RFC8372] Bryant, S., Pignataro, C., Chen, M., Li, Z., and G. Mirsky, "MPLS Flow Identification Considerations", RFC 8372, DOI 10.17487/RFC8372, May 2018, . [RFC8662] Kini, S., Kompella, K., Sivabalan, S., Litkowski, S., Shakir, R., and J. Tantsura, "Entropy Label for Source Packet Routing in Networking (SPRING) Tunnels", RFC 8662, DOI 10.17487/RFC8662, December 2019, . [RFC8957] Bryant, S., Chen, M., Swallow, G., Sivabalan, S., and G. Mirsky, "Synonymous Flow Label Framework", RFC 8957, DOI 10.17487/RFC8957, January 2021, . Cheng, et al. Expires 16 December 2023 [Page 13] Internet-Draft Encap for MPLS PM with AMM June 2023 [RFC9197] Brockners, F., Ed., Bhandari, S., Ed., and T. Mizrahi, Ed., "Data Fields for In Situ Operations, Administration, and Maintenance (IOAM)", RFC 9197, DOI 10.17487/RFC9197, May 2022, . [RFC9326] Song, H., Gafni, B., Brockners, F., Bhandari, S., and T. Mizrahi, "In Situ Operations, Administration, and Maintenance (IOAM) Direct Exporting", RFC 9326, DOI 10.17487/RFC9326, November 2022, . Authors' Addresses Weiqiang Cheng China Mobile Beijing China Email: chengweiqiang@chinamobile.com Xiao Min (editor) ZTE Corp. Nanjing China Email: xiao.min2@zte.com.cn Tianran Zhou Huawei Beijing China Email: zhoutianran@huawei.com Jinyou Dai FiberHome Wuhan China Email: djy@fiberhome.com Yoav Peleg Broadcom United States of America Email: yoav.peleg@broadcom.com Cheng, et al. Expires 16 December 2023 [Page 14]