1. z.archive.alto-dev@lists.opendaylight.org
  2. Messages
   Date   
[release]ALTO M5 Status

Gao Kai <gaok12@...>
 

1. Code Freeze (bug fixes only from here as defined above)
Yes

2. Stability branch, i.e., stable/lithium, must be cut and local project versions bumped on master to avoid overwriting lithium SNAPSHOTS
Stability branch has been cut.
Version has been bumped on master: https://git.opendaylight.org/gerrit/#/c/20468/

3. String Freeze (all externally visible strings frozen to allow for translation & documentation)
Yes

4. Documentation Complete: Only editing and and enhancing should take place after this point.
Yes, https://git.opendaylight.org/gerrit/20475

5. Integration & System Test
Features: https://git.opendaylight.org/gerrit/#/c/20383/
Test suites for odl-alto-provider and odl-alto-hosttracker feature: https://git.opendaylight.org/gerrit/#/c/20455/

TODO in M5 and future plans

Gao Kai
 

Hi all,

I’m going to write the documentation of ALTO within two or three days and we would be ready for M5.

After the CoNEXT we may get back to alto-0.2.0, and I’d like to share some thoughts on the components
and the corresponding functionalities.

  1. alto-core: The service abstractions and the road map to all ALTO instances.

    • alto-model: a new YANG model for ALTO services
    • alto-provider: declares ALTO service and provides some general functionalities such as identifier (resource id,
      endpoint id, etc.) management

  2. alto-basic: basic functionalities and helper functions/components

    • alto-simple-XXX: simple implementation of ALTO services: manually configurable network/cost map,
      network/cost map-based ecs/eps/filtered map/ird
    • alto-cli: karaf command line tool for alto-simple-xxx
    • alto-northbound: exploit ALTO service in RFC7285-format
    • …

    p.s. Lots of functionalities can be imported from palto.

  3. alto-extension: automatically generated, high-performance, enterprise-oriented ALTO services

    • hosttracker-based networkmap/costmap/ecs/eps
    • bgp-based networkmap/costmap/ecs/eps
    • …

  4. alto-applications: some other extra features that are based on ALTO

    • …

We may discuss the details and make plans for future development later.

Thanks,

Kai

​
Re: TODO in M5 and future plans

Y. Richard Yang
 

Kai,

Good list. In particular, I am interested in the -extension aspect. I am hoping that we have more ways to generate maps. We sure can discuss after this milestone.

Great work as the lead!

Richard

_____________________________

From: Gao Kai <godrickk@...>
Sent: Monday, May 18, 2015 4:28 PM
Subject: [alto-dev]TODO in M5 and future plans
To: <alto-dev@...>, Xin Wang (Tongji) <xinwang2014@...>, Richard <yry@...>, 董舒 <dongs2011@...>, Xin Li <yakumolx@...>


Hi all,

I’m going to write the documentation of ALTO within two or three days and we would be ready for M5.

After the CoNEXT we may get back to alto-0.2.0, and I’d like to share some thoughts on the components
and the corresponding functionalities.

  1. alto-core: The service abstractions and the road map to all ALTO instances.

    • alto-model: a new YANG model for ALTO services
    • alto-provider: declares ALTO service and provides some general functionalities such as identifier (resource id,
      endpoint id, etc.) management

  2. alto-basic: basic functionalities and helper functions/components

    • alto-simple-XXX: simple implementation of ALTO services: manually configurable network/cost map,
      network/cost map-based ecs/eps/filtered map/ird
    • alto-cli: karaf command line tool for alto-simple-xxx
    • alto-northbound: exploit ALTO service in RFC7285-format
    • …

    p.s. Lots of functionalities can be imported from palto.

  3. alto-extension: automatically generated, high-performance, enterprise-oriented ALTO services

    • hosttracker-based networkmap/costmap/ecs/eps
    • bgp-based networkmap/costmap/ecs/eps
    • …

  4. alto-applications: some other extra features that are based on ALTO

    • …

We may discuss the details and make plans for future development later.

Thanks,

Kai

​


Re: [alto] Interop test

Y. Richard Yang
 

Wendy,

Good proposal! Two comments:

1. I liked that the interop is for a setting that is close to real-life deployment. In particular, my understanding of the proposal is that the interop should leave as much unspecified as possible. A first reaction then is how to validate the correctness. For example, the response of resource-id is not known ahead of time. I assume that then the interop participants will need a second channel (e.g., human in the loop) to verify that a client c1 gets the correct response from a server s1. Is this the intended setting?

2. A second comment is that it will be interesting if the interop design will also become a starting point of ALTO compliance tests. We partition the tests into MUST and optional so that a test case is a MUST case if and only if (iff) it is a MUST in RFC7285. In the current proposal below, I see some mix (e.g., some services in the interop is not a MUST in RFC7285). Make sense?

Richard

On Fri, May 29, 2015 at 10:24 AM, Wendy Roome <w.roome@...> wrote:
Folks,

Rather than simply update the last test, I suggest a new approach,
outlined below.

**** General ****

The interop test defines two network maps: the default network map and an
alternate network map. We specify the PIDs and CIDRs for each map. For
each network map, the interop test defines the numerical-mode values for
the routingcost and hopcount cost-metrics. The interop test does not
define ordinal values for those metrics. Instead, each ALTO server may
assign ordinal values however it wants, as long as the ordinal values
preserve the numerical values.

The interop test also defines a custom endpoint property, say
"priv:ietf-type", and specifies the values.


**** Approach & Philosophy ****

Rather than specifying every detail, as in the previous interop test, this
iteration only specifies what is absolutely necessary. That is, the PIDs
in the network maps, the costs between PIDs, and the enpdoint property
values. Everything else -- resource ids, uris, cost-type names, etc. -- is
up to the server. Thus each server is completely characterized by two
parameters:

* The URI of its IRD.
* The resource id of its alternate map (if provided).

When a client is paired with a server, the client is given those two
parameters. The client is expected to determine everything else it needs
(e.g., the URIs of the default network map and the numerical routingcost
cost map) from the IRD.

Client-writers are encouraged to script their client so that it validates
the server's responses. That is, rather than just printing a cost map, the
client is encouraged to verify that it has the expected values. The
interop test will specify the maps in enough detail that clients will know
exactly what the server should return for any request.

Server-writers are encouraged to configure their server (especially the
IRD) so as to stress-test clients. That is, do things which are legal, but
unusual. For example, instead of having all resources in one IRD, the
server might have several linked (cascaded) IRDs. Or instead of offering
one filtered cost map, which handles all four combinations of cost metrics
& modes, plus constraints, the server might offer four separate filtered
cost map resources: one which handles numerical mode routingcost &
hopcount, but does not accept constraints, a second that handles the same
cost types, but which does accept constraints, and two more for ordinal
mode costs. This would verify that a client is able to select the correct
resource for a given request.


**** Server Requirements ****

Each server MUST offer the following resources:
* A full network map for the default network.
* Four full cost maps for the default network map: the routingcost &
hopcount metrics, in both numerical & ordinal modes.
* An endpoint property service for the pid property for the default
network map.

Each server MAY offer the following optional services:
* A filtered network map for the default network.
* A filtered cost map for the default network, optionally allowing cost
constraints.
* An endpoint cost service, covering the 4 cost metric/mode combinations,
optionally allowing cost constraints. The values must be derived from the
default network & cost maps.
* The filtered cost map & endpoint cost services may support constraints.
* An alternate network map plus the 4 associated full cost maps.
* An endpoint property service for the alternate network's pid property.
This may be the same EPS resource as for the default network, or a
different resource.
* A filtered network map for the alternate network.
* A filtered cost map for the alternate network map.

If offered, the filtered cost map & endpoint cost resources MUST cover all
four metric/mode combinations.

If offered, the costs for the endpoint cost service MUST must be derived
from the default network map.


**** Client Requirements ****

When a client is paired with a server, the client is given the URI of the
server's IRD, and the resource id of its alternate map (if the server &
client support alternate maps). The client must be able to extract all
other information from the IRD -- full map resources, their URIs, etc.

Each client MUST:

* Fetch the IRD.
* Locate & fetch the default network map and its four associated full cost
maps.
* Locate the EPS for the default network map's pid resource, and determine
the pid for a set of endpoints specified by the interop test.

Note that clients are not required to fetch the alternate network map (if
present). But every client must be able to parse an IRD with an alternate
network map, and distinguish the default network map, and its associate
resources, from the alternate map.

Optional client requirements:
* Use the filtered network map for the default network map to fetch PIDs
specified by the interop test.
* Use the filtered cost map for the default network map to fetch the costs
for several combinations of source pids, destination pids, metrics/modes,
and optional constraints, as specified by the interop test.
* Use the endpoint cost service to fetch the costs for several
combinations of source addresses, destination addresses, metrics/modes,
and optional constraints, as specified by the interop test.
* Locate & fetch the alternate map and its four full cost maps.
* Use the filtered network map for the alternate network map to fetch PIDs
specified by the interop test.
* Use the filtered cost map for the alternate network map to fetch the
costs for several combinations of source pids, destination pids,
metrics/modes, and optional constraints, as specified by the interop test.
* Fetch the custom endpoint property for a set of endpoints defined by the
interop test.


**** Error Testing ****

We will develop "torture test" scripts of bad client requests, and feed
them to each server to make sure the server gives reasonable error
responses. Participants are encouraged to contribute bad client requests
up to the time of the test. We will not specify the error tests in
advance, but will publish them after the interop is complete.

Note that the emphasis in on verifying that servers survive client errors.
It does not seem important to verify that clients survive server errors.
If you disagree, speak now or forever hold your peace.

    - Wendy Roome


_______________________________________________
alto mailing list
alto@...
https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_alto&d=AwICAg&c=-dg2m7zWuuDZ0MUcV7Sdqw&r=4G36iiEVb2m_v-0RnP2gx9KZJjYQgfvrOCE3789JGIA&m=kXIGnWFHVmEKkrU-3jzE_hUpglOe7Bj7H4Ys164dZBE&s=eXl0C-Yj1ZVYLA6lRFfpWyP1zKA5InHCoayBzITypGQ&e=
Re: [alto] Interop test

Wendy Roome <w.roome@...>
 

I was assuming the interop test would specify the network & cost maps. E.g., the pid names, their CIDRs, and the costs between PIDs. Given that, and given the IRD entry for a resource (e.g., its media-type, accepts, uses and capabilities), a validating client should know exactly what that resource will return.

Okay, there are a few exceptions, such as the tag for a network map and the values of an ordinal cost map. But a client can verify that the network map vtag in a cost map matches the vtag in the network map, and it can verify that ordinal cost values are consistent with the order of the known numerical values.

Operationally, the ideal validating client would print the results from the server, followed by a polite "C'est bon" or a big "I DISAGREE!!!"  

- Wendy

From: "Y. Richard Yang" <yry@...>
Date: Fri, May 29, 2015 at 11:40
To: Wendy Roome <w.roome@...>
Cc: "alto@..." <alto@...>, "Bertz, Lyle T [CTO]" <Lyle.T.Bertz@...>, Hans Seidel <hseidel@...>, "alto-dev@..." <alto-dev@...>
Subject: Re: [alto] Interop test

1. I liked that the interop is for a setting that is close to real-life deployment. In particular, my understanding of the proposal is that the interop should leave as much unspecified as possible. A first reaction then is how to validate the correctness. For example, the response of resource-id is not known ahead of time. I assume that then the interop participants will need a second channel (e.g., human in the loop) to verify that a client c1 gets the correct response from a server s1. Is this the intended setting?
Re: [alto] Interop test

Y. Richard Yang
 

Wendy,

On Fri, May 29, 2015 at 12:11 PM, Wendy Roome <w.roome@...> wrote:
I was assuming the interop test would specify the network & cost maps. E.g., the pid names, their CIDRs, and the costs between PIDs. Given that, and given the IRD entry for a resource (e.g., its media-type, accepts, uses and capabilities), a validating client should know exactly what that resource will return.

Okay, there are a few exceptions, such as the tag for a network map and the values of an ordinal cost map. But a client can verify that the network map vtag in a cost map matches the vtag in the network map,

This makes sense. We can specify the validation condition for the interop:

- For network map, the only unspecified value then is the "vtag" field, and the interop specifies the assert, which is based on consistency between data and IRD.

- For cost map, the only unspecified is the exact "dependent-vtags", and your validation makes sense.

 
and it can verify that ordinal cost values are consistent with the order of the known numerical values.

This is a reasonable validation. An issue is that RFC7285 specifies only that "An ALTO server MUST support at least one of the following modes:  numerical and ordinal." I believe that RFC7285 chose to not specify the consistency between the two modes. Hence, this is beyond compliance, right? Hence, I feel that separating RFC7285-conforming and beyond can be helpful.

Richard

 

Operationally, the ideal validating client would print the results from the server, followed by a polite "C'est bon" or a big "I DISAGREE!!!"  

- Wendy

From: "Y. Richard Yang" <yry@...>
Date: Fri, May 29, 2015 at 11:40
To: Wendy Roome <w.roome@...>
Cc: "alto@..." <alto@...>, "Bertz, Lyle T [CTO]" <Lyle.T.Bertz@...>, Hans Seidel <hseidel@...>, "alto-dev@..." <alto-dev@...>
Subject: Re: [alto] Interop test

1. I liked that the interop is for a setting that is close to real-life deployment. In particular, my understanding of the proposal is that the interop should leave as much unspecified as possible. A first reaction then is how to validate the correctness. For example, the response of resource-id is not known ahead of time. I assume that then the interop participants will need a second channel (e.g., human in the loop) to verify that a client c1 gets the correct response from a server s1. Is this the intended setting?
Re: [alto] Interop test

Wendy Roome <w.roome@...>
 


But I have not seen IETF compliance tests for other protocols. Does the IETF officially sponsor/endorse them? Or must they be done outside of the IETF?

- Wendy Roome

From: "Y. Richard Yang" <yry@...>
Date: Fri, May 29, 2015 at 11:40
To: Wendy Roome <w.roome@...>
Cc: "alto@..." <alto@...>, "Bertz, Lyle T [CTO]" <Lyle.T.Bertz@...>, Hans Seidel <hseidel@...>, "alto-dev@..." <alto-dev@...>
Subject: Re: [alto] Interop test

2. A second comment is that it will be interesting if the interop design will also become a starting point of ALTO compliance tests. We partition the tests into MUST and optional so that a test case is a MUST case if and only if (iff) it is a MUST in RFC7285. In the current proposal below, I see some mix (e.g., some services in the interop is not a MUST in RFC7285). Make sense?
Re: [alto] Interop test

Y. Richard Yang
 


On Fri, May 29, 2015 at 12:26 PM, Wendy Roome <w.roome@...> wrote:

Super cool!
 
But I have not seen IETF compliance tests for other protocols. Does the IETF officially sponsor/endorse them? Or must they be done outside of the IETF?


I guess this is a question that the chairs/AD can provide the guidance :-)

Richard

 
- Wendy Roome

From: "Y. Richard Yang" <yry@...>
Date: Fri, May 29, 2015 at 11:40
To: Wendy Roome <w.roome@...>
Cc: "alto@..." <alto@...>, "Bertz, Lyle T [CTO]" <Lyle.T.Bertz@...>, Hans Seidel <hseidel@...>, "alto-dev@..." <alto-dev@...>
Subject: Re: [alto] Interop test

2. A second comment is that it will be interesting if the interop design will also become a starting point of ALTO compliance tests. We partition the tests into MUST and optional so that a test case is a MUST case if and only if (iff) it is a MUST in RFC7285. In the current proposal below, I see some mix (e.g., some services in the interop is not a MUST in RFC7285). Make sense?
Re: [alto] Interop test

Wendy Roome <w.roome@...>
 

I thought RFC 7285 required order-consistency between numerical & ordinal modes for the same metric. But I cannot find that requirement. Too bad! I would have added that if I realized it wasn't there.

But if we take off our lawyer hats, and look at RFC 7285 as an agreement between friends, rather than a formal contract between adversaries, then I think it is reasonable to say that if the ordinal mode costs do not preserve the order of the numerical costs, then that server is wrong.

Similarly, a router is allowed to drop packets, and I do not think there is any formal requirement that it cannot drop *every* packet. So theoretically you could glue eight jacks to a block of hardwood and market it as a router. Maybe you could avoid getting charged for fraud. Just don't expect anyone to buy more than one! :-)

And looking at it from a real-world perspective, the whole concept is irrelevant. The only reason for defining ordinal mode is to allow a server to hide the numerical costs. For example, if the numerical costs to three PIDs are 10, 11 and 100, a client can deduce that the middle pid is close. If the costs are 10, 99 and 100, a client can deduce that the middle pid is distant. If the ordinal costs are 1,2,3, a client cannot deduce anything other than 1 is better than 2 is better than 3.

So if a server offers numerical costs, there is no advantage for it to also offer ordinal mode costs.

And if numerical costs are available, there is no advantage to a client to use ordinal costs. Maybe if the client could assume the ordinal costs are 1,2,3,… -- but the client cannot.

So in practice, no server will offer both numerical & ordinal mode for the same metric.

That means a formal compliance test should only require one mode or the other, but not both. However, keep the interop simple, unless someone objects, I suggest we require both modes. 

- Wendy

From: "Y. Richard Yang" <yry@...>
Date: Fri, May 29, 2015 at 12:24
To: Wendy Roome <w.roome@...>
Cc: "alto@..." <alto@...>, "Bertz, Lyle T [CTO]" <Lyle.T.Bertz@...>, Hans Seidel <hseidel@...>, "alto-dev@..." <alto-dev@...>
Subject: Re: [alto] Interop test

 
and it can verify that ordinal cost values are consistent with the order of the known numerical values.

This is a reasonable validation. An issue is that RFC7285 specifies only that "An ALTO server MUST support at least one of the following modes:  numerical and ordinal." I believe that RFC7285 chose to not specify the consistency between the two modes. Hence, this is beyond compliance, right? Hence, I feel that separating RFC7285-conforming and beyond can be helpful.
Lithium RC0 Status

Thanh Ha <thanh.ha@...>
 

FYI

---------- Forwarded message ----------
From: Thanh Ha <thanh.ha@...>
Date: 29 May 2015 at 13:51
Subject: Lithium RC0 Status
To: "release@..." <release@...>


Hi Everyone,

A few folks have pinged me about this so I thought I'd send out an email. I've spent most of my day working out and submitting patches for building a successful Lithium RC0 and projects have been pretty quick at responding so far.

Unfortunately I'm now blocked on moving forward with an Alto patch that needs to be merged [1]. Alto's Karaf bundle's is pulling down the Beryllium release of Controller's karaf-parent. Of course this does not exist in Lithium so the build is failing and unable to find that artifact. I hope the Alto project can merge this patch soon so we can make more progress on getting an RC0 out asap.

Regards,

Thanh
Re: [alto] Interop test

Y. Richard Yang
 



On Saturday, May 30, 2015, Wendy Roome <w.roome@...> wrote:
I thought RFC 7285 required order-consistency between numerical & ordinal modes for the same metric. But I cannot find that requirement. Too bad! I would have added that if I realized it wasn't there.

I remembered that the removal of the consistency requirement was a conscious decision, not an omission, to reduce the load of consistency checking. I buy your point below of a server supporting only one, in the sense of providing the maximum allowed by the privacy concern.

Regarding supporting both in the interop, I will be happy to wait a bit to hear others' opinions.

Richard

But if we take off our lawyer hats, and look at RFC 7285 as an agreement between friends, rather than a formal contract between adversaries, then I think it is reasonable to say that if the ordinal mode costs do not preserve the order of the numerical costs, then that server is wrong.

Similarly, a router is allowed to drop packets, and I do not think there is any formal requirement that it cannot drop *every* packet. So theoretically you could glue eight jacks to a block of hardwood and market it as a router. Maybe you could avoid getting charged for fraud. Just don't expect anyone to buy more than one! :-)

And looking at it from a real-world perspective, the whole concept is irrelevant. The only reason for defining ordinal mode is to allow a server to hide the numerical costs. For example, if the numerical costs to three PIDs are 10, 11 and 100, a client can deduce that the middle pid is close. If the costs are 10, 99 and 100, a client can deduce that the middle pid is distant. If the ordinal costs are 1,2,3, a client cannot deduce anything other than 1 is better than 2 is better than 3.

So if a server offers numerical costs, there is no advantage for it to also offer ordinal mode costs.

And if numerical costs are available, there is no advantage to a client to use ordinal costs. Maybe if the client could assume the ordinal costs are 1,2,3,… -- but the client cannot.

So in practice, no server will offer both numerical & ordinal mode for the same metric.

That means a formal compliance test should only require one mode or the other, but not both. However, keep the interop simple, unless someone objects, I suggest we require both modes. 

- Wendy

From: "Y. Richard Yang" <yry@...>
Date: Fri, May 29, 2015 at 12:24
To: Wendy Roome <w.roome@...>
Cc: "alto@..." <alto@...>, "Bertz, Lyle T [CTO]" <Lyle.T.Bertz@...>, Hans Seidel <hseidel@...>, "alto-dev@..." <alto-dev@...>
Subject: Re: [alto] Interop test

 
and it can verify that ordinal cost values are consistent with the order of the known numerical values.

This is a reasonable validation. An issue is that RFC7285 specifies only that "An ALTO server MUST support at least one of the following modes:  numerical and ordinal." I believe that RFC7285 chose to not specify the consistency between the two modes. Hence, this is beyond compliance, right? Hence, I feel that separating RFC7285-conforming and beyond can be helpful.


--
Richard
Re: [alto] Interop test

Bertz, Lyle T [CTO] <Lyle.T.Bertz@...>
 

Wendy,

 

On Friday, May 29, 2015, Wendy Roome <w.roome@...> wrote:

“Similarly, a router is allowed to drop packets, and I do not think there is any formal requirement that it cannot drop *every* packet. So theoretically you could glue eight jacks to a block of hardwood and market it as a router. Maybe you could avoid getting charged for fraud. Just don't expect anyone to buy more than one! :-)”

 

I am pretty sure someone has actually tried to sell me one of these and can find it in the lab (but was not the purchaser).  To date its loss rate is close to 100%.  It is treated more like a very secure firewall than a router and is placed beside a tin pale that is a bit bucket someone purchased long ago. :D

 

On a serious note, I agree with you that only one mode would be supported on the server .   Please note that an ordinal ranking of a metric may exist if there are too many entries that have the same value in the original metric.  We do this quite often as an operator but we *always* give this metric a different name in order to emphasize the ranking aspect.  

 

I have a much larger question about ordinal rank in 7285, is it the expectation that ordinal ranks are true metrics in practice.  In other words are they or even the original metrics true mathematical metrics, i.e. non-negative, have symmetry, coincidence axiom and the triangle inequality.  Further are they formally ultrametrics or intrinsic?  

 

I only ask because depending on the answer we can add services for max distance, sum distance and the like that apply graph theory to the resulting maps.  Who know, I may even be able to apply my only graph theories to them but I will keep such aspirations low at the moment.

 

Lyle

 

PS – Are we looking to perform an interop in July or the next meeting?

 

From: Wendy Roome [mailto:w.roome@...]
Sent: Friday, May 29, 2015 12:06 PM
To: Y. Richard Yang
Cc: alto@...; Bertz, Lyle T [CTO]; Hans Seidel; alto-dev@...
Subject: Re: [alto] Interop test

 

I thought RFC 7285 required order-consistency between numerical & ordinal modes for the same metric. But I cannot find that requirement. Too bad! I would have added that if I realized it wasn't there.

 

But if we take off our lawyer hats, and look at RFC 7285 as an agreement between friends, rather than a formal contract between adversaries, then I think it is reasonable to say that if the ordinal mode costs do not preserve the order of the numerical costs, then that server is wrong.

 

Similarly, a router is allowed to drop packets, and I do not think there is any formal requirement that it cannot drop *every* packet. So theoretically you could glue eight jacks to a block of hardwood and market it as a router. Maybe you could avoid getting charged for fraud. Just don't expect anyone to buy more than one! :-)

 

And looking at it from a real-world perspective, the whole concept is irrelevant. The only reason for defining ordinal mode is to allow a server to hide the numerical costs. For example, if the numerical costs to three PIDs are 10, 11 and 100, a client can deduce that the middle pid is close. If the costs are 10, 99 and 100, a client can deduce that the middle pid is distant. If the ordinal costs are 1,2,3, a client cannot deduce anything other than 1 is better than 2 is better than 3.

 

So if a server offers numerical costs, there is no advantage for it to also offer ordinal mode costs.

 

And if numerical costs are available, there is no advantage to a client to use ordinal costs. Maybe if the client could assume the ordinal costs are 1,2,3,… -- but the client cannot.

 

So in practice, no server will offer both numerical & ordinal mode for the same metric.

 

That means a formal compliance test should only require one mode or the other, but not both. However, keep the interop simple, unless someone objects, I suggest we require both modes. 

 

                - Wendy

 

From: "Y. Richard Yang" <yry@...>
Date: Fri, May 29, 2015 at 12:24
To: Wendy Roome <w.roome@...>
Cc: "alto@..." <alto@...>, "Bertz, Lyle T [CTO]" <Lyle.T.Bertz@...>, Hans Seidel <hseidel@...>, "alto-dev@..." <alto-dev@...>
Subject: Re: [alto] Interop test

 

 

and it can verify that ordinal cost values are consistent with the order of the known numerical values.

 

This is a reasonable validation. An issue is that RFC7285 specifies only that "An ALTO server MUST support at least one of the following modes:  numerical and ordinal." I believe that RFC7285 chose to not specify the consistency between the two modes. Hence, this is beyond compliance, right? Hence, I feel that separating RFC7285-conforming and beyond can be helpful.



This e-mail may contain Sprint proprietary information intended for the sole use of the recipient(s). Any use by others is prohibited. If you are not the intended recipient, please contact the sender and delete all copies of the message.

Re: [alto] Interop test

Wendy Roome <w.roome@...>
 

Interesting point!  RFC 7285 does NOT require cost metrics -- numerical or ordinal -- to follow the requirements for a formal metric. Other than the values must be non-negative.

In particular, there is no requirement that d(x,x) = 0, or that d(x,y) = 0 iff x = y.

Costs are directed, so symmetry isn't even appropriate.

I guess I would expect a numeric metric to follow the triangle inequality, more or less, but there is no formal requirement for it do so.

Incidentally, for ordinal mode costs, the values are only comparable to other costs in the same request. In other words, if an ordinal cost in a filtered cost map is 0, that just means it is the lowest cost for the set of sources & destinations you requested. It is NOT the lowest cost for the full map.

- Wendy Roome


From: "Bertz, Lyle T [CTO]" <Lyle.T.Bertz@...>
Date: Fri, May 29, 2015 at 15:21
To: Wendy Roome <w.roome@...>, "Y. Richard Yang" <yry@...>
Cc: "alto@..." <alto@...>, Hans Seidel <hseidel@...>, "alto-dev@..." <alto-dev@...>
Subject: RE: [alto] Interop test

I have a much larger question about ordinal rank in 7285, is it the expectation that ordinal ranks are true metrics in practice.  In other words are they or even the original metrics true mathematical metrics, i.e. non-negative, have symmetry, coincidence axiom and the triangle inequality.  Further are they formally ultrametrics or intrinsic?  

 

I only ask because depending on the answer we can add services for max distance, sum distance and the like that apply graph theory to the resulting maps.  Who know, I may even be able to apply my only graph theories to them but I will keep such aspirations low at the moment.
Re: [alto] Interop test

Bertz, Lyle T [CTO] <Lyle.T.Bertz@...>
 

I had not put together the comparison applying to the same requests.   That makes matters convenient as the included data only applies to the response. 

 

Without formal metric compliance though there are a lot of algorithms that could not be reliably applied.  I think we should consider this as we move forward in developing ALTO.  It may be worthwhile to clarify matters in a future update.

 

From: Wendy Roome [mailto:w.roome@...]
Sent: Friday, May 29, 2015 2:45 PM
To: Bertz, Lyle T [CTO]; Y. Richard Yang
Cc: alto@...; Hans Seidel; alto-dev@...
Subject: Re: [alto] Interop test

 

Interesting point!  RFC 7285 does NOT require cost metrics -- numerical or ordinal -- to follow the requirements for a formal metric. Other than the values must be non-negative.

 

In particular, there is no requirement that d(x,x) = 0, or that d(x,y) = 0 iff x = y.

 

Costs are directed, so symmetry isn't even appropriate.

 

I guess I would expect a numeric metric to follow the triangle inequality, more or less, but there is no formal requirement for it do so.

 

Incidentally, for ordinal mode costs, the values are only comparable to other costs in the same request. In other words, if an ordinal cost in a filtered cost map is 0, that just means it is the lowest cost for the set of sources & destinations you requested. It is NOT the lowest cost for the full map.

 

                - Wendy Roome

 

 

From: "Bertz, Lyle T [CTO]" <Lyle.T.Bertz@...>
Date: Fri, May 29, 2015 at 15:21
To: Wendy Roome <w.roome@...>, "Y. Richard Yang" <yry@...>
Cc: "alto@..." <alto@...>, Hans Seidel <hseidel@...>, "alto-dev@..." <alto-dev@...>
Subject: RE: [alto] Interop test

 

I have a much larger question about ordinal rank in 7285, is it the expectation that ordinal ranks are true metrics in practice.  In other words are they or even the original metrics true mathematical metrics, i.e. non-negative, have symmetry, coincidence axiom and the triangle inequality.  Further are they formally ultrametrics or intrinsic?  

 

I only ask because depending on the answer we can add services for max distance, sum distance and the like that apply graph theory to the resulting maps.  Who know, I may even be able to apply my only graph theories to them but I will keep such aspirations low at the moment.





This e-mail may contain Sprint proprietary information intended for the sole use of the recipient(s). Any use by others is prohibited. If you are not the intended recipient, please contact the sender and delete all copies of the message.

Re: [alto] Interop test

Wendy Roome <w.roome@...>
 

Lyle,

I haven't used metric spaces since the days of keypunches, so it took a while to blow the cobwebs out of that memory bank. :-)

I don't think ALTO cost metrics could ever meet the requirements for a metric space. The problems:

* A true metric d(x,y) must be defined for all x & y. ALTO does not require that the cost be defined for all pairs.

* For a metric, d(x,y) = d(y,x) for all x,y. ALTO costs represent communication links, and are potentially asymmetric. Eg,  the download bandwidth can be higher than the upload bandwidth.

* For a metric, d(x,x) = 0 for all x. For ALTO, the cost within a PID frequently is greater than 0.

* For a metric, d(x,y) = 0 means x = y. We *might* be able to define ALTO costs so that is possible, but I have my doubts.

* For a metric, d(x,z) <= d(x,y) + d(y,z) for all y. For ALTO, that is tempting requirement, and I cannot think of an obvious counter example. But I doubt that would help without the other requirements.

So we could not require *every* ALTO cost metric to qualify as a mathematical metric. But we could define a particular cost as satisfying those rules, and (say) identify it in the IRD.  Can you explain how that would benefit clients or servers? One obvious advantage is that the cost matrix could be smaller, because its symmetric and the center diagonal is 0.

- Wendy Roome

From: "Bertz, Lyle T [CTO]" <Lyle.T.Bertz@...>
Date: Fri, May 29, 2015 at 16:03
To: Wendy Roome <w.roome@...>, "Y. Richard Yang" <yry@...>
Cc: "alto@..." <alto@...>, Hans Seidel <hseidel@...>, "alto-dev@..." <alto-dev@...>
Subject: RE: [alto] Interop test

I had not put together the comparison applying to the same requests.   That makes matters convenient as the included data only applies to the response. 

 

Without formal metric compliance though there are a lot of algorithms that could not be reliably applied.  I think we should consider this as we move forward in developing ALTO.  It may be worthwhile to clarify matters in a future update.

 

From: Wendy Roome [mailto:w.roome@...]
Sent: Friday, May 29, 2015 2:45 PM
To: Bertz, Lyle T [CTO]; Y. Richard Yang
Cc: alto@...; Hans Seidel; alto-dev@...
Subject: Re: [alto] Interop test

 

Interesting point!  RFC 7285 does NOT require cost metrics -- numerical or ordinal -- to follow the requirements for a formal metric. Other than the values must be non-negative.

 

In particular, there is no requirement that d(x,x) = 0, or that d(x,y) = 0 iff x = y.

 

Costs are directed, so symmetry isn't even appropriate.

 

I guess I would expect a numeric metric to follow the triangle inequality, more or less, but there is no formal requirement for it do so.

 

Incidentally, for ordinal mode costs, the values are only comparable to other costs in the same request. In other words, if an ordinal cost in a filtered cost map is 0, that just means it is the lowest cost for the set of sources & destinations you requested. It is NOT the lowest cost for the full map.

 

                - Wendy Roome

 

 

From: "Bertz, Lyle T [CTO]" <Lyle.T.Bertz@...>
Date: Fri, May 29, 2015 at 15:21
To: Wendy Roome <w.roome@...>, "Y. Richard Yang" <yry@...>
Cc: "alto@..." <alto@...>, Hans Seidel <hseidel@...>, "alto-dev@..." <alto-dev@...>
Subject: RE: [alto] Interop test

 

I have a much larger question about ordinal rank in 7285, is it the expectation that ordinal ranks are true metrics in practice.  In other words are they or even the original metrics true mathematical metrics, i.e. non-negative, have symmetry, coincidence axiom and the triangle inequality.  Further are they formally ultrametrics or intrinsic?  

 

I only ask because depending on the answer we can add services for max distance, sum distance and the like that apply graph theory to the resulting maps.  Who know, I may even be able to apply my only graph theories to them but I will keep such aspirations low at the moment.





This e-mail may contain Sprint proprietary information intended for the sole use of the recipient(s). Any use by others is prohibited. If you are not the intended recipient, please contact the sender and delete all copies of the message.
FW: [release] version bump progress

George Zhao <George.Y.Zhao@...>
 

Hi Richard, Vijay,

 

Could you take care of the two patches.

 

Thanks,

George

 

From: Colin Dixon [mailto:colin@...]
Sent: Monday, June 01, 2015 12:19 PM
To: George Zhao
Cc: Thanh Ha; release@...
Subject: Re: [release] version bump progress

 

I'm looking at this and it seems that there are two projects in a dangerous spot where they have a stable/lithium branch but have not bumped versions on their master branch:

* ALTO: please merge https://git.opendaylight.org/gerrit/#/c/18332/

* SDNi: please merge: https://git.opendaylight.org/gerrit/#/c/19765/

If we could have both those patches merged, that would help.

 

--Colin

 

 

On Mon, Jun 1, 2015 at 12:11 PM, George Zhao <George.Y.Zhao@...> wrote:

Hello,

 

Every project should have completed stable/lithium branch cutting and version bump,  please do so ASAP if your project hasn’t completed the two tasks indicate in Thanh’s email [1].

 

Thanks,

George

 

From: Thanh Ha [mailto:thanh.ha@...]
Sent: Friday, May 29, 2015 12:31 PM
To: George Zhao; release@...
Subject: Re: version bump progress

 

Hi Everyone,

 

I updated the Lithium Status spreadsheet to add 2 additional tabs "Branched" and "Version Bumped" to track whether a project has branched, and if they version bumped. This should make it easier for everyone to keep updated on which projects are still waiting to be bumped or branched.

 

I noticed a few projects already have version bump patches submitted weren't merged because they were failing. I'll go and run rechecks on all of them now that openflowplugin has branched. Hopefully some of these projects can proceed with merging their version bump patches now.

 

Regards,

 

Thanh

 


_______________________________________________
release mailing list
release@...
https://lists.opendaylight.org/mailman/listinfo/release

 
Re: FW: [release] version bump progress

Y. Richard Yang
 

George,

Sorry for missing this. We (Kai/Shu) will take care of it right away.

Thanks for the reminder!
Richard

_____________________________

From: George Zhao <george.y.zhao@...>
Sent: Tuesday, June 2, 2015 5:56 AM
Subject: FW: [release] version bump progress
To: <vanandr@...>, <yry@...>
Cc: <snbi-dev@...>, <alto-dev@...>


Hi Richard, Vijay,

 

Could you take care of the two patches.

 

Thanks,

George

 

From: Colin Dixon [mailto:colin@...]
Sent: Monday, June 01, 2015 12:19 PM
To: George Zhao
Cc: Thanh Ha; release@...
Subject: Re: [release] version bump progress

 

I'm looking at this and it seems that there are two projects in a dangerous spot where they have a stable/lithium branch but have not bumped versions on their master branch:

* ALTO: please merge https://git.opendaylight.org/gerrit/#/c/18332/

* SDNi: please merge: https://git.opendaylight.org/gerrit/#/c/19765/

If we could have both those patches merged, that would help.

 

--Colin

 

 

On Mon, Jun 1, 2015 at 12:11 PM, George Zhao <George.Y.Zhao@...> wrote:

Hello,

 

Every project should have completed stable/lithium branch cutting and version bump,  please do so ASAP if your project hasn’t completed the two tasks indicate in Thanh’s email [1].

 

Thanks,

George

 

From: Thanh Ha [mailto:thanh.ha@...]
Sent: Friday, May 29, 2015 12:31 PM
To: George Zhao; release@...
Subject: Re: version bump progress

 

Hi Everyone,

 

I updated the Lithium Status spreadsheet to add 2 additional tabs "Branched" and "Version Bumped" to track whether a project has branched, and if they version bumped. This should make it easier for everyone to keep updated on which projects are still waiting to be bumped or branched.

 

I noticed a few projects already have version bump patches submitted weren't merged because they were failing. I'll go and run rechecks on all of them now that openflowplugin has branched. Hopefully some of these projects can proceed with merging their version bump patches now.

 

Regards,

 

Thanh

 


_______________________________________________
release mailing list
release@...
https://lists.opendaylight.org/mailman/listinfo/release

 
Re: FW: [release] version bump progress

George Zhao <George.Y.Zhao@...>
 

Thanks

George

 

From: Yale [mailto:yry@...]
Sent: Monday, June 01, 2015 7:01 PM
To: George Zhao
Cc: alto-dev@...
Subject: Re: FW: [release] version bump progress

 

George,

 

Sorry for missing this. We (Kai/Shu) will take care of it right away.

 

Thanks for the reminder!
Richard

 

_____________________________
From: George Zhao <george.y.zhao@...>
Sent: Tuesday, June 2, 2015 5:56 AM
Subject: FW: [release] version bump progress
To: <vanandr@...>, <yry@...>
Cc: <snbi-dev@...>, <alto-dev@...>

Hi Richard, Vijay,

 

Could you take care of the two patches.

 

Thanks,

George

 

From: Colin Dixon [mailto:colin@...]
Sent: Monday, June 01, 2015 12:19 PM
To: George Zhao
Cc: Thanh Ha; release@...
Subject: Re: [release] version bump progress

 

I'm looking at this and it seems that there are two projects in a dangerous spot where they have a stable/lithium branch but have not bumped versions on their master branch:

* ALTO: please merge https://git.opendaylight.org/gerrit/#/c/18332/

* SDNi: please merge: https://git.opendaylight.org/gerrit/#/c/19765/

If we could have both those patches merged, that would help.

 

--Colin

 

 

On Mon, Jun 1, 2015 at 12:11 PM, George Zhao <George.Y.Zhao@...> wrote:

Hello,

 

Every project should have completed stable/lithium branch cutting and version bump,  please do so ASAP if your project hasn’t completed the two tasks indicate in Thanh’s email [1].

 

Thanks,

George

 

From: Thanh Ha [mailto:thanh.ha@...]
Sent: Friday, May 29, 2015 12:31 PM
To: George Zhao; release@...
Subject: Re: version bump progress

 

Hi Everyone,

 

I updated the Lithium Status spreadsheet to add 2 additional tabs "Branched" and "Version Bumped" to track whether a project has branched, and if they version bumped. This should make it easier for everyone to keep updated on which projects are still waiting to be bumped or branched.

 

I noticed a few projects already have version bump patches submitted weren't merged because they were failing. I'll go and run rechecks on all of them now that openflowplugin has branched. Hopefully some of these projects can proceed with merging their version bump patches now.

 

Regards,

 

Thanh

 


_______________________________________________
release mailing list
release@...
https://lists.opendaylight.org/mailman/listinfo/release

 

 
Re: [alto] Interop test

Hans Seidel <hseidel@...>
 

Richard

On 29.05.2015 20:24, Y. Richard Yang wrote:


On Saturday, May 30, 2015, Wendy Roome <w.roome@...> wrote:
I thought RFC 7285 required order-consistency between numerical & ordinal modes for the same metric. But I cannot find that requirement. Too bad! I would have added that if I realized it wasn't there.

I remembered that the removal of the consistency requirement was a conscious decision, not an omission, to reduce the load of consistency checking. I buy your point below of a server supporting only one, in the sense of providing the maximum allowed by the privacy concern.

Regarding supporting both in the interop, I will be happy to wait a bit to hear others' opinions.

I agree that providing both numerical and ordinal mode for the same cost metric makes little sense from a real world perspective. For the interop, I think both should be covered. Since Wendy already proposed cost maps for routingcost and hopcount in her initial mail, I suggest providing a numerical cost map for one metric and an ordinal for the other.

Hans


Richard

But if we take off our lawyer hats, and look at RFC 7285 as an agreement between friends, rather than a formal contract between adversaries, then I think it is reasonable to say that if the ordinal mode costs do not preserve the order of the numerical costs, then that server is wrong.

Similarly, a router is allowed to drop packets, and I do not think there is any formal requirement that it cannot drop *every* packet. So theoretically you could glue eight jacks to a block of hardwood and market it as a router. Maybe you could avoid getting charged for fraud. Just don't expect anyone to buy more than one! :-)

And looking at it from a real-world perspective, the whole concept is irrelevant. The only reason for defining ordinal mode is to allow a server to hide the numerical costs. For example, if the numerical costs to three PIDs are 10, 11 and 100, a client can deduce that the middle pid is close. If the costs are 10, 99 and 100, a client can deduce that the middle pid is distant. If the ordinal costs are 1,2,3, a client cannot deduce anything other than 1 is better than 2 is better than 3.

So if a server offers numerical costs, there is no advantage for it to also offer ordinal mode costs.

And if numerical costs are available, there is no advantage to a client to use ordinal costs. Maybe if the client could assume the ordinal costs are 1,2,3,… -- but the client cannot.

So in practice, no server will offer both numerical & ordinal mode for the same metric.

That means a formal compliance test should only require one mode or the other, but not both. However, keep the interop simple, unless someone objects, I suggest we require both modes. 

- Wendy

From: "Y. Richard Yang" <yry@...>
Date: Fri, May 29, 2015 at 12:24
To: Wendy Roome <w.roome@...>
Cc: "alto@..." <alto@...>, "Bertz, Lyle T [CTO]" <Lyle.T.Bertz@...>, Hans Seidel <hseidel@...>, "alto-dev@..." <alto-dev@...>
Subject: Re: [alto] Interop test

 
and it can verify that ordinal cost values are consistent with the order of the known numerical values.

This is a reasonable validation. An issue is that RFC7285 specifies only that "An ALTO server MUST support at least one of the following modes:  numerical and ordinal." I believe that RFC7285 chose to not specify the consistency between the two modes. Hence, this is beyond compliance, right? Hence, I feel that separating RFC7285-conforming and beyond can be helpful.


--
Richard
Re: [alto] Interop test

Y. Richard Yang
 



On Tuesday, June 2, 2015, Hans Seidel <hseidel@...> wrote:
Richard

On 29.05.2015 20:24, Y. Richard Yang wrote:

On Saturday, May 30, 2015, Wendy Roome <w.roome@...> wrote:
I thought RFC 7285 required order-consistency between numerical & ordinal modes for the same metric. But I cannot find that requirement. Too bad! I would have added that if I realized it wasn't there.

I remembered that the removal of the consistency requirement was a conscious decision, not an omission, to reduce the load of consistency checking. I buy your point below of a server supporting only one, in the sense of providing the maximum allowed by the privacy concern.

Regarding supporting both in the interop, I will be happy to wait a bit to hear others' opinions.

I agree that providing both numerical and ordinal mode for the same cost metric makes little sense from a real world perspective. For the interop, I think both should be covered. Since Wendy already proposed cost maps for routingcost and hopcount in her initial mail, I suggest providing a numerical cost map for one metric and an ordinal for the other. 

 This proposal makes sense to me. I support it.

Richard 


Hans


Richard

But if we take off our lawyer hats, and look at RFC 7285 as an agreement between friends, rather than a formal contract between adversaries, then I think it is reasonable to say that if the ordinal mode costs do not preserve the order of the numerical costs, then that server is wrong.

Similarly, a router is allowed to drop packets, and I do not think there is any formal requirement that it cannot drop *every* packet. So theoretically you could glue eight jacks to a block of hardwood and market it as a router. Maybe you could avoid getting charged for fraud. Just don't expect anyone to buy more than one! :-)

And looking at it from a real-world perspective, the whole concept is irrelevant. The only reason for defining ordinal mode is to allow a server to hide the numerical costs. For example, if the numerical costs to three PIDs are 10, 11 and 100, a client can deduce that the middle pid is close. If the costs are 10, 99 and 100, a client can deduce that the middle pid is distant. If the ordinal costs are 1,2,3, a client cannot deduce anything other than 1 is better than 2 is better than 3.

So if a server offers numerical costs, there is no advantage for it to also offer ordinal mode costs.

And if numerical costs are available, there is no advantage to a client to use ordinal costs. Maybe if the client could assume the ordinal costs are 1,2,3,… -- but the client cannot.

So in practice, no server will offer both numerical & ordinal mode for the same metric.

That means a formal compliance test should only require one mode or the other, but not both. However, keep the interop simple, unless someone objects, I suggest we require both modes. 

- Wendy

From: "Y. Richard Yang" <yry@...>
Date: Fri, May 29, 2015 at 12:24
To: Wendy Roome <w.roome@...>
Cc: "alto@..." <alto@...>, "Bertz, Lyle T [CTO]" <Lyle.T.Bertz@...>, Hans Seidel <hseidel@...>, "alto-dev@..." <alto-dev@...>
Subject: Re: [alto] Interop test

 
and it can verify that ordinal cost values are consistent with the order of the known numerical values.

This is a reasonable validation. An issue is that RFC7285 specifies only that "An ALTO server MUST support at least one of the following modes:  numerical and ordinal." I believe that RFC7285 chose to not specify the consistency between the two modes. Hence, this is beyond compliance, right? Hence, I feel that separating RFC7285-conforming and beyond can be helpful.


--
Richard





--
Richard
81 - 100 of 542