z.archive.ttp-dev@lists.opendaylight.org | [release] Beryllium System Test Waiver Requests

Home Messages Hashtags Subgroups

This group is locked. No changes can be made to the group while it is locked.

Date Date 1 - 4 of 4

[release] Beryllium System Test Waiver Requests

Jamo Luhrsen <jluhrsen@...> #108 Hi Curt, We had a discussion about all projects requesting System Test Waivers in our last integration call, and I wanted to double back to TTP to clarify some things before we approve the waiver. In the TTP M2 Status [0] there is mention that there are external APIs to be consumed and that there would be top-level features that require system test. If this is the case, then we don't need a wavier, right? Currently, there doesn't appear to be any TTP suites in our test repo. Can you clarify what the plan is going forward for Beryllium? Thanks, JamO [0] https://lists.opendaylight.org/pipermail/release/2015-September/003674.html toggle quoted message Show quoted text On 09/18/2015 08:57 AM, Curt Beckmann wrote: Thanks, Vratko. I do not expect that the utility piece of the TTP project would be part of the Beryllium release, but will discuss within the ttp-dev community… If we seek to make it part of the release, then we will pursue the system test work. Curt From: release-bounces@... [mailto:release-bounces@...] On Behalf Of Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) Sent: Friday, September 18, 2015 4:58 PM To: An Ho; integration-dev@...; Release Subject: Re: [release] Beryllium System Test Waiver Requests Quick reactions: TTP: Some features of the TTP project are part of a standalone (CLI utility) executable, which would not be tested as part of system test. Is the utility part of Beryllium release? If not, no problem. Projects are not prohibited from having outputs unrelated to the release. If yes, it is a part of larger ODL system and should be system tested. Messaging4Transport: The project's integration test covers testing the data flow through MD-SAL. System tests are there also for verifying other features (odl-integration-compatible-with-all) do not interfere. A simple analogue to some RESTCONF system test would be appreciated. OpFlex: OpFlex is a non-Java project; tests are run in build as unit test and also external testing External tests are promised, no reason not to Approve. The same for SNMP4SDN. NETCONF: No Explanation. Um. Netconf system tests description is still used as an example [1] and there are integration/test contributors already working on them [2]. I really see no reason to Approve this waiver. All other projects in list: Approve. Vratko. [1] https://wiki.opendaylight.org/view/CrossProject:Integration_Group:Feature_Integration_System_Test_Template#Examples_From_The_Lithium_Release [2] https://git.opendaylight.org/gerrit/#/c/26505 From:integration-dev-bounces@... <mailto:integration-dev-bounces@...> [mailto:integration-dev-bounces@...] On Behalf Of An Ho Sent: Wednesday, September 16, 2015 10:36 PM To: integration-dev@... <mailto:integration-dev@...>; Release Subject: [integration-dev] Beryllium System Test Waiver Requests Hi Integration Team, Based on M2 status, 11 projects have requested system test waivers. Their requests are summarized in this wiki page here [1]. Please take moment to review and update the status of each request to “Acknowledged” or “Approved” on the wiki page. Please let me know if you have questions. Best Regards, An Ho [1] https://wiki.opendaylight.org/view/Simultaneous_Release/Beryllium/Waiver/System_Test [2] https://docs.google.com/spreadsheets/d/1Kfp5HVQGydNegEjp6dD2V3XsUnqpice0bysWbdxABA4/edit#gid=818929996 _______________________________________________ release mailing list release@... https://lists.opendaylight.org/mailman/listinfo/release
Curt Beckmann #109 Hi Jamo, We do need to get some tests into the test repo, as you say. The question of the waiver is a bit separate... Our project can also generate a separate executable (we call it the "parser") which leverages some of the YANG modeling in ODL to do some manipulation and analysis of TTPs. The ultimate purpose of this parser tool is to construct something (maybe a JSON file?) for importation into the full ODL controller. So, the "parser" is used by developers. The parser has interfaces, but we expect to test those interfaces separately from the system test. I guess there is some question whether we even need a waiver for this...but that was the point of discussing the waiver. On the other front, we are looking at whether we can achieve the functionality we want through integration with DIDM, and without API changes on either side. However it ends up, we will need to put the tests into the repo, as you say. Does that provide enough clarification? Curt toggle quoted message Show quoted text -----Original Message----- From: Jamo Luhrsen [mailto:jluhrsen@...] Sent: Tuesday, September 29, 2015 6:21 PM To: Curt Beckmann; Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco); An Ho; integration-dev@...; Release; ttp- dev@... Subject: Re: [release] Beryllium System Test Waiver Requests Hi Curt, We had a discussion about all projects requesting System Test Waivers in our last integration call, and I wanted to double back to TTP to clarify some things before we approve the waiver. In the TTP M2 Status [0] there is mention that there are external APIs to be consumed and that there would be top-level features that require system test. If this is the case, then we don't need a wavier, right? Currently, there doesn't appear to be any TTP suites in our test repo. Can you clarify what the plan is going forward for Beryllium? Thanks, JamO [0] https://lists.opendaylight.org/pipermail/release/2015- September/003674.html On 09/18/2015 08:57 AM, Curt Beckmann wrote: Thanks, Vratko. I do not expect that the utility piece of the TTP project would be part of the Beryllium release, but will discuss within the ttp-dev community… If we seek to make it part of the release, then we will pursue the system test work. Curt From: release-bounces@... [mailto:release-bounces@...] On Behalf Of Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) Sent: Friday, September 18, 2015 4:58 PM To: An Ho; integration-dev@...; Release Subject: Re: [release] Beryllium System Test Waiver Requests Quick reactions: TTP: Some features of the TTP project are part of a standalone (CLI utility) executable, which would not be tested as part of system test. Is the utility part of Beryllium release? If not, no problem. Projects are not prohibited from having outputs unrelated to the release. If yes, it is a part of larger ODL system and should be system tested. Messaging4Transport: The project's integration test covers testing the data flow through MD- SAL. System tests are there also for verifying other features (odl-integration-compatible-with-all) do not interfere. A simple analogue to some RESTCONF system test would be appreciated. OpFlex: OpFlex is a non-Java project; tests are run in build as unit test and also external testing External tests are promised, no reason not to Approve. The same for SNMP4SDN. NETCONF: No Explanation. Um. Netconf system tests description is still used as an example [1] and there are integration/test contributors already working on them [2]. I really see no reason to Approve this waiver. All other projects in list: Approve. Vratko. [1] https://wiki.opendaylight.org/view/CrossProject:Integration_Group:Feat ure_Integration_System_Test_Template#Examples_From_The_Lithium_Rel ease [2] https://git.opendaylight.org/gerrit/#/c/26505 From:integration-dev-bounces@... <mailto:integration-dev-bounces@...> [mailto:integration-dev-bounces@...] On Behalf Of An Ho Sent: Wednesday, September 16, 2015 10:36 PM To: integration-dev@... <mailto:integration-dev@...>; Release Subject: [integration-dev] Beryllium System Test Waiver Requests Hi Integration Team, Based on M2 status, 11 projects have requested system test waivers. Their requests are summarized in this wiki page here [1]. Please take moment to review and update the status of each request to “Acknowledged” or “Approved” on the wiki page. Please let me know if you have questions. Best Regards, An Ho [1] https://wiki.opendaylight.org/view/Simultaneous_Release/Beryllium/Waiv er/System_Test [2] https://docs.google.com/spreadsheets/d/1Kfp5HVQGydNegEjp6dD2V3XsU nqpic e0bysWbdxABA4/edit#gid=818929996 _______________________________________________ release mailing list release@... https://lists.opendaylight.org/mailman/listinfo/release
Jamo Luhrsen <jluhrsen@...> #110 Hi Curt, I've never forced myself to truly understand TTP before, so trying now. inline... On 09/30/2015 03:16 AM, Curt Beckmann wrote: Hi Jamo, We do need to get some tests into the test repo, as you say. The question of the waiver is a bit separate... Our project can also generate a separate executable (we call it the "parser") which leverages some of the YANG modeling in ODL to do some manipulation and analysis of TTPs. The ultimate purpose of this parser tool is to construct something (maybe a JSON file?) for importation into the full ODL controller. So, the "parser" is used by developers. I can see some system tests here, even for a tool to only be used by developers. We could execute the workflow you describe in a CSIT job, right? manipulate/analyze TTPs --> generate "something" --> import to ODL --> validate The parser has interfaces, but we expect to test those interfaces separately from the system test. I guess there is some question whether we even need a waiver for this...but that was the point of discussing the waiver. So, yeah if the parser, as a standalone executable, is tested elsewhere somehow then no problem with a waiver for that. On the other front, we are looking at whether we can achieve the functionality we want through integration with DIDM, and without API changes on either side. However it ends up, we will need to put the tests into the repo, as you say. This is something you are still working through now, I take it. We can worry about system tests for that if/when it becomes a reality. Does that provide enough clarification? I think so. My opinion for system tests requirements for an ODL release is that if something is being added to the controller distribution as an end-user accessible feature, then we need system tests automated and running for those. For outside-the-box (e.g. the parser tool) things, then I would still encourage upstream tests but we don't have any requirement spelled out for that. If the parser is the only Be deliverable, then we grant the waiver. Public APIs on the ODL side coming from the TTP project (are there?) would need system test, so we don't give the waiver. what do you think? JamO Curt -----Original Message----- From: Jamo Luhrsen [mailto:jluhrsen@...] Sent: Tuesday, September 29, 2015 6:21 PM To: Curt Beckmann; Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco); An Ho; integration-dev@...; Release; ttp- dev@... Subject: Re: [release] Beryllium System Test Waiver Requests Hi Curt, We had a discussion about all projects requesting System Test Waivers in our last integration call, and I wanted to double back to TTP to clarify some things before we approve the waiver. In the TTP M2 Status [0] there is mention that there are external APIs to be consumed and that there would be top-level features that require system test. If this is the case, then we don't need a wavier, right? Currently, there doesn't appear to be any TTP suites in our test repo. Can you clarify what the plan is going forward for Beryllium? Thanks, JamO [0] https://lists.opendaylight.org/pipermail/release/2015- September/003674.html On 09/18/2015 08:57 AM, Curt Beckmann wrote: Thanks, Vratko. I do not expect that the utility piece of the TTP project would be part of the Beryllium release, but will discuss within the ttp-dev community… If we seek to make it part of the release, then we will pursue the system test work. Curt From: release-bounces@... [mailto:release-bounces@...] On Behalf Of Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) Sent: Friday, September 18, 2015 4:58 PM To: An Ho; integration-dev@...; Release Subject: Re: [release] Beryllium System Test Waiver Requests Quick reactions: TTP: Some features of the TTP project are part of a standalone (CLI utility) executable, which would not be tested as part of system test. Is the utility part of Beryllium release? If not, no problem. Projects are not prohibited from having outputs unrelated to the release. If yes, it is a part of larger ODL system and should be system tested. Messaging4Transport: The project's integration test covers testing the data flow through MD- SAL. System tests are there also for verifying other features (odl-integration-compatible-with-all) do not interfere. A simple analogue to some RESTCONF system test would be appreciated. OpFlex: OpFlex is a non-Java project; tests are run in build as unit test and also external testing External tests are promised, no reason not to Approve. The same for SNMP4SDN. NETCONF: No Explanation. Um. Netconf system tests description is still used as an example [1] and there are integration/test contributors already working on them [2]. I really see no reason to Approve this waiver. All other projects in list: Approve. Vratko. [1] https://wiki.opendaylight.org/view/CrossProject:Integration_Group:Feat ure_Integration_System_Test_Template#Examples_From_The_Lithium_Rel ease [2] https://git.opendaylight.org/gerrit/#/c/26505 From:integration-dev-bounces@... <mailto:integration-dev-bounces@...> [mailto:integration-dev-bounces@...] On Behalf Of An Ho Sent: Wednesday, September 16, 2015 10:36 PM To: integration-dev@... <mailto:integration-dev@...>; Release Subject: [integration-dev] Beryllium System Test Waiver Requests Hi Integration Team, Based on M2 status, 11 projects have requested system test waivers. Their requests are summarized in this wiki page here [1]. Please take moment to review and update the status of each request to “Acknowledged” or “Approved” on the wiki page. Please let me know if you have questions. Best Regards, An Ho [1] https://wiki.opendaylight.org/view/Simultaneous_Release/Beryllium/Waiv er/System_Test [2] https://docs.google.com/spreadsheets/d/1Kfp5HVQGydNegEjp6dD2V3XsU nqpic e0bysWbdxABA4/edit#gid=818929996 _______________________________________________ release mailing list release@... https://lists.opendaylight.org/mailman/listinfo/release
Curt Beckmann #111 Hi Jamo, Sorry for the lag. I think this all makes sense. Seems like the waiver is not appropriate... I thought perhaps it made sense for the decoupled piece (the standalone parser) but that's not the case. Curt toggle quoted message Show quoted text -----Original Message----- From: Jamo Luhrsen [mailto:jluhrsen@...] Sent: Thursday, October 01, 2015 7:49 AM To: Curt Beckmann; Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco); An Ho; integration-dev@...; Release; ttp- dev@... Subject: Re: [release] Beryllium System Test Waiver Requests Hi Curt, I've never forced myself to truly understand TTP before, so trying now. inline... On 09/30/2015 03:16 AM, Curt Beckmann wrote: Hi Jamo, We do need to get some tests into the test repo, as you say. The question of the waiver is a bit separate... Our project can also generate a separate executable (we call it the "parser") which leverages some of the YANG modeling in ODL to do some manipulation and analysis of TTPs. The ultimate purpose of this parser tool is to construct something (maybe a JSON file?) for importation into the full ODL controller. So, the "parser" is used by developers. I can see some system tests here, even for a tool to only be used by developers. We could execute the workflow you describe in a CSIT job, right? manipulate/analyze TTPs --> generate "something" --> import to ODL --> validate The parser has interfaces, but we expect to test those interfaces separately from the system test. I guess there is some question whether we even need a waiver for this...but that was the point of discussing the waiver. So, yeah if the parser, as a standalone executable, is tested elsewhere somehow then no problem with a waiver for that. On the other front, we are looking at whether we can achieve the functionality we want through integration with DIDM, and without API changes on either side. However it ends up, we will need to put the tests into the repo, as you say. This is something you are still working through now, I take it. We can worry about system tests for that if/when it becomes a reality. Does that provide enough clarification? I think so. My opinion for system tests requirements for an ODL release is that if something is being added to the controller distribution as an end-user accessible feature, then we need system tests automated and running for those. For outside-the-box (e.g. the parser tool) things, then I would still encourage upstream tests but we don't have any requirement spelled out for that. If the parser is the only Be deliverable, then we grant the waiver. Public APIs on the ODL side coming from the TTP project (are there?) would need system test, so we don't give the waiver. what do you think? JamO Curt -----Original Message----- From: Jamo Luhrsen [mailto:jluhrsen@...] Sent: Tuesday, September 29, 2015 6:21 PM To: Curt Beckmann; Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco); An Ho; integration-dev@...; Release; ttp- dev@... Subject: Re: [release] Beryllium System Test Waiver Requests Hi Curt, We had a discussion about all projects requesting System Test Waivers in our last integration call, and I wanted to double back to TTP to clarify some things before we approve the waiver. In the TTP M2 Status [0] there is mention that there are external APIs to be consumed and that there would be top-level features that require system test. If this is the case, then we don't need a wavier, right? Currently, there doesn't appear to be any TTP suites in our test repo. Can you clarify what the plan is going forward for Beryllium? Thanks, JamO [0] https://lists.opendaylight.org/pipermail/release/2015- September/003674.html On 09/18/2015 08:57 AM, Curt Beckmann wrote: Thanks, Vratko. I do not expect that the utility piece of the TTP project would be part of the Beryllium release, but will discuss within the ttp-dev community… If we seek to make it part of the release, then we will pursue the system test work. Curt From: release-bounces@... [mailto:release-bounces@...] On Behalf Of Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) Sent: Friday, September 18, 2015 4:58 PM To: An Ho; integration-dev@...; Release Subject: Re: [release] Beryllium System Test Waiver Requests Quick reactions: TTP: Some features of the TTP project are part of a standalone (CLI utility) executable, which would not be tested as part of system test. Is the utility part of Beryllium release? If not, no problem. Projects are not prohibited from having outputs unrelated to the release. If yes, it is a part of larger ODL system and should be system tested. Messaging4Transport: The project's integration test covers testing the data flow through MD- SAL. System tests are there also for verifying other features (odl-integration-compatible-with-all) do not interfere. A simple analogue to some RESTCONF system test would be appreciated. OpFlex: OpFlex is a non-Java project; tests are run in build as unit test and also external testing External tests are promised, no reason not to Approve. The same for SNMP4SDN. NETCONF: No Explanation. Um. Netconf system tests description is still used as an example [1] and there are integration/test contributors already working on them [2]. I really see no reason to Approve this waiver. All other projects in list: Approve. Vratko. [1] https://wiki.opendaylight.org/view/CrossProject:Integration_Group:Fe at ure_Integration_System_Test_Template#Examples_From_The_Lithium_Rel ease [2] https://git.opendaylight.org/gerrit/#/c/26505 From:integration-dev-bounces@... <mailto:integration-dev-bounces@...> [mailto:integration-dev-bounces@...] On Behalf Of An Ho Sent: Wednesday, September 16, 2015 10:36 PM To: integration-dev@... <mailto:integration-dev@...>; Release Subject: [integration-dev] Beryllium System Test Waiver Requests Hi Integration Team, Based on M2 status, 11 projects have requested system test waivers. Their requests are summarized in this wiki page here [1]. Please take moment to review and update the status of each request to “Acknowledged” or “Approved” on the wiki page. Please let me know if you have questions. Best Regards, An Ho [1] https://wiki.opendaylight.org/view/Simultaneous_Release/Beryllium/Wa iv er/System_Test [2] https://docs.google.com/spreadsheets/d/1Kfp5HVQGydNegEjp6dD2V3XsU nqpic e0bysWbdxABA4/edit#gid=818929996 _______________________________________________ release mailing list release@... https://lists.opendaylight.org/mailman/listinfo/release

1 - 4 of 4

1

Previous Topic Next Topic

Home Hashtags Subgroups Terms