[controller-dev] First shot at serializing data tree entities using the Binding Independent level


Colin Dixon
 

Sorry I haven't caught up with this. Did you make any more progress?

--Colin


On Tue, Aug 23, 2016 at 5:54 AM, Sela, Guy <guy.sela@...> wrote:

+yangtools mailing list

 

From: Sela, Guy
Sent: Tuesday, August 23, 2016 12:30 PM
To: 'Robert Varga' <nite@...>; 'mdsal-dev@....org' <mdsal-dev@....org>; 'controller-dev@lists.opendaylight.org' <controller-dev@lists.opendaylight.org>; 'ttp-dev@....org' <ttp-dev@....org>
Subject: RE: First shot at serializing data tree entities using the Binding Independent level

 

Regarding my first question for creating the QName in the receiver side, I found that I can use:

BindingReflections.findQName(ElanInstances.class).

So theoretically, if I were to send the Class type along with the Json string, I can deserialize everything in the receiver side without any “knowledge”.

 

From: Sela, Guy
Sent: Tuesday, August 23, 2016 12:18 PM
To: 'Robert Varga' <nite@...>; mdsal-dev@....org; controller-dev@lists.opendaylight.org; 'ttp-dev@....org' <ttp-dev@....org>
Subject: First shot at serializing data tree entities using the Binding Independent level

 

Hi,

 

As suggested, I looked at the TTPUtils for an example on how to serialize and deserialize the DTOs between machines.

My attempt was to serialize an ElanInstance from the elan.yang model.

This is the model:

container elan-instances {

        list elan-instance {

               ….

 

First of all, serializing the ElanInstance itself resulted in:

Exception in thread "main" java.lang.IllegalArgumentException: List item is not appropriate

               at com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)

               at org.opendaylight.yangtools.yang.data.impl.codec.SchemaTracker.startListItem(SchemaTracker.java:144)

               at org.opendaylight.yangtools.yang.data.codec.gson.JSONNormalizedNodeStreamWriter.startMapEntryNode(JSONNormalizedNodeStreamWriter.java:156)

               at org.opendaylight.yangtools.binding.data.codec.impl.BindingToNormalizedStreamWriter.startMapEntryNode(BindingToNormalizedStreamWriter.java:175)

               at org.opendaylight.yang.gen.v1.urn.opendaylight.netvirt.elan.rev150602.elan.instances.ElanInstance$StreamWriter.serialize(DataObjectSerializerPrototype.java)

               at org.opendaylight.yangtools.binding.data.codec.impl.BindingNormalizedNodeCodecRegistry$DataObjectSerializerProxy.serialize(BindingNormalizedNodeCodecRegistry.java:295)

               at org.opendaylight.newfederation.transform.TTPUtils.jsonStringFromDataObject(TTPUtils.java:146)

               at org.opendaylight.newfederation.tests.JsonParsing.main(JsonParsing.java:73)

 

Then, I tried to serialize the ElanInstances container initialized with the ElanInstance, and it succeeded, but with a lot of verbose code, that I guess could be reduced somehow.

This is the code:

SERIALIZER CODE:

        TTPUtils ttp = new TTPUtils(Collections.singleton(BindingReflections.getModuleInfo(ElanInstances.class)));

        ElanInstance newInstance = new ElanInstanceBuilder().setElanInstanceName("GUYELAN")

                .setDescription("ElanDescription").build();

        ArrayList<ElanInstance> list = new ArrayList<>();

        list.add(newInstance);

        ElanInstances instances = new ElanInstancesBuilder().setElanInstance(list).build();

        InstanceIdentifier<ElanInstances> id = InstanceIdentifier.create(ElanInstances.class);

 

        String jsonStringFromDataObject = ttp.jsonStringFromDataObject(id, instances, true);

        System.out.println(jsonStringFromDataObject);

DESERIALIZER CODE:

        NormalizedNode<?, ?> normalized = ttp.normalizedNodeFromJsonString(jsonStringFromDataObject);

        DataObject finalDataObject = ttp.dataObjectFromNormalizedNode("urn:opendaylight:netvirt:elan", "2015-06-02",

                "elan-instances", normalized);

        ElanInstances finalInstances = (ElanInstances) finalDataObject;

        System.out.println("RESULT: " + finalInstances.getElanInstance().get(0).getElanInstanceName());

 

 

I implemented the ttp.dataObjectFromNormalizedNode myself, like this:

public DataObject dataObjectFromNormalizedNode(String namespace, String revision, String localName, NormalizedNode<?, ?> nn) {

        QName qname = QName.create(namespace, revision, localName);

        return codecRegistry.fromNormalizedNode(YangInstanceIdentifier.of(qname), nn).getValue();

    }

 

Now to my questions:

1)      Is it possible to do what I’m trying to do with less “knowledge” in the deserializer side?  The deserializer had to know that the NormalizedNode it got was specifically belongs to this model:  ("urn:opendaylight:netvirt:elan", "2015-06-02", "elan-instances"), in order to do the deserialization.

2)      How will this work with augmentations that are declared in different models?

3)      Can I use it with a specific ElanInstance? (I got an exception as I illustrated).

4)      I have a feeling I am not doing this in the right way, is there any example of how to implement what I’m trying to do? If not, can someone help me with a code snippet ?

 

Thanks,

Guy Sela

 


_______________________________________________
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev



Sela, Guy <guy.sela@...>
 

Yeah, this is what I done:

The only way it is working for me now, is the following (Some of it is pseudo-code):

"

PRODUCER CODE:

I have a org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.topology.Node in my hands (Got it from a DCN).

This is the code I need to run in order to do what I'm trying:

TopologyBuilder topologyBuilder = new TopologyBuilder();

topologyBuilder.setKey(new TopologyKey(new TopologyId(new Uri("ovsdb:1"))));

topologyBuilder.setNode(Collections.singletonList(node));

NetworkTopologyBuilder ntBuilder = new NetworkTopologyBuilder();

ntBuilder.setTopology(Collections.singletonList(topologyBuilder.build()));

InstanceIdentifier path = InstanceIdentifier.create(NetworkTopology.class);

String json = TTPUtils.jsonStringFromDataObject(path, ntBuilder.build(), true);

Serialize the json...

 

CONSUMER CODE:

NormalizedNode normalizedNode = TTPUtils.normalizedNodeFromJsonString(jsonInput);

DataObject obj = BindingNormalizedNodeCodecRegistry. fromNormalizedNode(YangInstanceIdentifier.of(BindingReflections.findQName(NetworkTopology.class)), normalizedNode);

NetworkTopology nt = (NetworkTopology)obj;

Node node = nt.getTopology().get(0).getNode().get(0);

return node;

"

 

If I want to manipulate the data before sending, it looks nastier:

 

EGRESS EXAMPLE (1): // Changing networkId before sending it on the wire

PortBuilder pb = new PortBuilder(port);

pb.setNetworkId(federatedNetworkUid);

return pb.build();

 

INGRESS EXAMPLE (1): // Changing networkId, tenantId, subnetId

Neutron neutron = (Neutron) input.getInput();

Port port = neutron.getPorts().getPort().get(0);

PortBuilder pb = new PortBuilder(port);

LOG.info("Manipulating from Network ID: " + pb.getNetworkId() + " to Network ID: " + localNetworkUid);

pb.setNetworkId(localNetworkUid);

LOG.info("Manipulating from Tenant ID: " + pb.getTenantId() + " to Tenant ID: " + localTenantUid);

pb.setTenantId(localTenantUid);

List<FixedIps> federatedFixedIps = pb.getFixedIps();

if (federatedFixedIps != null && !federatedFixedIps.isEmpty()) {

List<FixedIps> localFixedIps = new ArrayList<>();

federatedFixedIps.forEach(ip -> localFixedIps.add(new FixedIpsBuilder(ip).setSubnetId(localSubnetUid).build()));

pb.setFixedIps(localFixedIps);

}

return pb.build();

 

EGRESS EXAMPLE (2): // Removing unnecessary augmentations before sending

NodeBuilder nodeBuilder = new NodeBuilder(node);

nodeBuilder.addAugmentation(FlowCapableNode.class, null);

nodeBuilder.addAugmentation(FlowCapableStatisticsGatheringStatus.class, null);

List<NodeConnector> ncList = node.getNodeConnector();

List<NodeConnector> newNcList = new ArrayList<>();

if (ncList != null) {

for (NodeConnector nc : ncList) {

NodeConnectorBuilder ncBuilder = new NodeConnectorBuilder(nc);

             ncBuilder.addAugmentation(FlowCapableNodeConnectorStatisticsData.class, null);

                              newNcList.add(ncBuilder.build());

}

}

nodeBuilder.setNodeConnector(newNcList);

node = nodeBuilder.build();

 

 

 

The manipulations code is very not elegant as an understatement, given the current Binding API.

 

From: Colin Dixon [mailto:colin@...]
Sent: Wednesday, September 07, 2016 9:30 PM
To: Sela, Guy <guy.sela@...>
Cc: Robert Varga <nite@...>; mdsal-dev@...; controller-dev@...; ttp-dev@...; yangtools-dev@...
Subject: Re: [controller-dev] First shot at serializing data tree entities using the Binding Independent level

 

Sorry I haven't caught up with this. Did you make any more progress?

--Colin

 

On Tue, Aug 23, 2016 at 5:54 AM, Sela, Guy <guy.sela@...> wrote:

+yangtools mailing list

 

From: Sela, Guy
Sent: Tuesday, August 23, 2016 12:30 PM
To: 'Robert Varga' <nite@...>; 'mdsal-dev@...' <mdsal-dev@...>; 'controller-dev@...' <controller-dev@...>; 'ttp-dev@...' <ttp-dev@...>
Subject: RE: First shot at serializing data tree entities using the Binding Independent level

 

Regarding my first question for creating the QName in the receiver side, I found that I can use:

BindingReflections.findQName(ElanInstances.class).

So theoretically, if I were to send the Class type along with the Json string, I can deserialize everything in the receiver side without any “knowledge”.

 

From: Sela, Guy
Sent: Tuesday, August 23, 2016 12:18 PM
To: 'Robert Varga' <nite@...>; mdsal-dev@...; controller-dev@...; 'ttp-dev@...' <ttp-dev@...>
Subject: First shot at serializing data tree entities using the Binding Independent level

 

Hi,

 

As suggested, I looked at the TTPUtils for an example on how to serialize and deserialize the DTOs between machines.

My attempt was to serialize an ElanInstance from the elan.yang model.

This is the model:

container elan-instances {

        list elan-instance {

               ….

 

First of all, serializing the ElanInstance itself resulted in:

Exception in thread "main" java.lang.IllegalArgumentException: List item is not appropriate

               at com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)

               at org.opendaylight.yangtools.yang.data.impl.codec.SchemaTracker.startListItem(SchemaTracker.java:144)

               at org.opendaylight.yangtools.yang.data.codec.gson.JSONNormalizedNodeStreamWriter.startMapEntryNode(JSONNormalizedNodeStreamWriter.java:156)

               at org.opendaylight.yangtools.binding.data.codec.impl.BindingToNormalizedStreamWriter.startMapEntryNode(BindingToNormalizedStreamWriter.java:175)

               at org.opendaylight.yang.gen.v1.urn.opendaylight.netvirt.elan.rev150602.elan.instances.ElanInstance$StreamWriter.serialize(DataObjectSerializerPrototype.java)

               at org.opendaylight.yangtools.binding.data.codec.impl.BindingNormalizedNodeCodecRegistry$DataObjectSerializerProxy.serialize(BindingNormalizedNodeCodecRegistry.java:295)

               at org.opendaylight.newfederation.transform.TTPUtils.jsonStringFromDataObject(TTPUtils.java:146)

               at org.opendaylight.newfederation.tests.JsonParsing.main(JsonParsing.java:73)

 

Then, I tried to serialize the ElanInstances container initialized with the ElanInstance, and it succeeded, but with a lot of verbose code, that I guess could be reduced somehow.

This is the code:

SERIALIZER CODE:

        TTPUtils ttp = new TTPUtils(Collections.singleton(BindingReflections.getModuleInfo(ElanInstances.class)));

        ElanInstance newInstance = new ElanInstanceBuilder().setElanInstanceName("GUYELAN")

                .setDescription("ElanDescription").build();

        ArrayList<ElanInstance> list = new ArrayList<>();

        list.add(newInstance);

        ElanInstances instances = new ElanInstancesBuilder().setElanInstance(list).build();

        InstanceIdentifier<ElanInstances> id = InstanceIdentifier.create(ElanInstances.class);

 

        String jsonStringFromDataObject = ttp.jsonStringFromDataObject(id, instances, true);

        System.out.println(jsonStringFromDataObject);

DESERIALIZER CODE:

        NormalizedNode<?, ?> normalized = ttp.normalizedNodeFromJsonString(jsonStringFromDataObject);

        DataObject finalDataObject = ttp.dataObjectFromNormalizedNode("urn:opendaylight:netvirt:elan", "2015-06-02",

                "elan-instances", normalized);

        ElanInstances finalInstances = (ElanInstances) finalDataObject;

        System.out.println("RESULT: " + finalInstances.getElanInstance().get(0).getElanInstanceName());

 

 

I implemented the ttp.dataObjectFromNormalizedNode myself, like this:

public DataObject dataObjectFromNormalizedNode(String namespace, String revision, String localName, NormalizedNode<?, ?> nn) {

        QName qname = QName.create(namespace, revision, localName);

        return codecRegistry.fromNormalizedNode(YangInstanceIdentifier.of(qname), nn).getValue();

    }

 

Now to my questions:

1)      Is it possible to do what I’m trying to do with less “knowledge” in the deserializer side?  The deserializer had to know that the NormalizedNode it got was specifically belongs to this model:  ("urn:opendaylight:netvirt:elan", "2015-06-02", "elan-instances"), in order to do the deserialization.

2)      How will this work with augmentations that are declared in different models?

3)      Can I use it with a specific ElanInstance? (I got an exception as I illustrated).

4)      I have a feeling I am not doing this in the right way, is there any example of how to implement what I’m trying to do? If not, can someone help me with a code snippet ?

 

Thanks,

Guy Sela

 


_______________________________________________
controller-dev mailing list
controller-dev@...
https://lists.opendaylight.org/mailman/listinfo/controller-dev

 


Colin Dixon
 

I probably just know too much about how ugly this was to start, but that looks pretty clean at this point.

--Colin


On Thu, Sep 8, 2016 at 3:01 AM, Sela, Guy <guy.sela@...> wrote:

Yeah, this is what I done:

The only way it is working for me now, is the following (Some of it is pseudo-code):

"

PRODUCER CODE:

I have a org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.topology.Node in my hands (Got it from a DCN).

This is the code I need to run in order to do what I'm trying:

TopologyBuilder topologyBuilder = new TopologyBuilder();

topologyBuilder.setKey(new TopologyKey(new TopologyId(new Uri("ovsdb:1"))));

topologyBuilder.setNode(Collections.singletonList(node));

NetworkTopologyBuilder ntBuilder = new NetworkTopologyBuilder();

ntBuilder.setTopology(Collections.singletonList(topologyBuilder.build()));

InstanceIdentifier path = InstanceIdentifier.create(NetworkTopology.class);

String json = TTPUtils.jsonStringFromDataObject(path, ntBuilder.build(), true);

Serialize the json...

 

CONSUMER CODE:

NormalizedNode normalizedNode = TTPUtils.normalizedNodeFromJsonString(jsonInput);

DataObject obj = BindingNormalizedNodeCodecRegistry. fromNormalizedNode(YangInstanceIdentifier.of(BindingReflections.findQName(NetworkTopology.class)), normalizedNode);

NetworkTopology nt = (NetworkTopology)obj;

Node node = nt.getTopology().get(0).getNode().get(0);

return node;

"

 

If I want to manipulate the data before sending, it looks nastier:

 

EGRESS EXAMPLE (1): // Changing networkId before sending it on the wire

PortBuilder pb = new PortBuilder(port);

pb.setNetworkId(federatedNetworkUid);

return pb.build();

 

INGRESS EXAMPLE (1): // Changing networkId, tenantId, subnetId

Neutron neutron = (Neutron) input.getInput();

Port port = neutron.getPorts().getPort().get(0);

PortBuilder pb = new PortBuilder(port);

LOG.info("Manipulating from Network ID: " + pb.getNetworkId() + " to Network ID: " + localNetworkUid);

pb.setNetworkId(localNetworkUid);

LOG.info("Manipulating from Tenant ID: " + pb.getTenantId() + " to Tenant ID: " + localTenantUid);

pb.setTenantId(localTenantUid);

List<FixedIps> federatedFixedIps = pb.getFixedIps();

if (federatedFixedIps != null && !federatedFixedIps.isEmpty()) {

List<FixedIps> localFixedIps = new ArrayList<>();

federatedFixedIps.forEach(ip -> localFixedIps.add(new FixedIpsBuilder(ip).setSubnetId(localSubnetUid).build()));

pb.setFixedIps(localFixedIps);

}

return pb.build();

 

EGRESS EXAMPLE (2): // Removing unnecessary augmentations before sending

NodeBuilder nodeBuilder = new NodeBuilder(node);

nodeBuilder.addAugmentation(FlowCapableNode.class, null);

nodeBuilder.addAugmentation(FlowCapableStatisticsGatheringStatus.class, null);

List<NodeConnector> ncList = node.getNodeConnector();

List<NodeConnector> newNcList = new ArrayList<>();

if (ncList != null) {

for (NodeConnector nc : ncList) {

NodeConnectorBuilder ncBuilder = new NodeConnectorBuilder(nc);

             ncBuilder.addAugmentation(FlowCapableNodeConnectorStatisticsData.class, null);

                              newNcList.add(ncBuilder.build());

}

}

nodeBuilder.setNodeConnector(newNcList);

node = nodeBuilder.build();

 

 

 

The manipulations code is very not elegant as an understatement, given the current Binding API.

 

From: Colin Dixon [mailto:colin@...]
Sent: Wednesday, September 07, 2016 9:30 PM
To: Sela, Guy <guy.sela@...>
Cc: Robert Varga <nite@...>; mdsal-dev@....org; controller-dev@lists.opendaylight.org; ttp-dev@...; yangtools-dev@lists.opendaylight.org
Subject: Re: [controller-dev] First shot at serializing data tree entities using the Binding Independent level

 

Sorry I haven't caught up with this. Did you make any more progress?

--Colin

 

On Tue, Aug 23, 2016 at 5:54 AM, Sela, Guy <guy.sela@...> wrote:

+yangtools mailing list

 

From: Sela, Guy
Sent: Tuesday, August 23, 2016 12:30 PM
To: 'Robert Varga' <nite@...>; 'mdsal-dev@....org' <mdsal-dev@....org>; 'controller-dev@lists.opendaylight.org' <controller-dev@lists.opendaylight.org>; 'ttp-dev@....org' <ttp-dev@....org>
Subject: RE: First shot at serializing data tree entities using the Binding Independent level

 

Regarding my first question for creating the QName in the receiver side, I found that I can use:

BindingReflections.findQName(ElanInstances.class).

So theoretically, if I were to send the Class type along with the Json string, I can deserialize everything in the receiver side without any “knowledge”.

 

From: Sela, Guy
Sent: Tuesday, August 23, 2016 12:18 PM
To: 'Robert Varga' <nite@...>; mdsal-dev@....org; controller-dev@lists.opendaylight.org; 'ttp-dev@....org' <ttp-dev@....org>
Subject: First shot at serializing data tree entities using the Binding Independent level

 

Hi,

 

As suggested, I looked at the TTPUtils for an example on how to serialize and deserialize the DTOs between machines.

My attempt was to serialize an ElanInstance from the elan.yang model.

This is the model:

container elan-instances {

        list elan-instance {

               ….

 

First of all, serializing the ElanInstance itself resulted in:

Exception in thread "main" java.lang.IllegalArgumentException: List item is not appropriate

               at com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)

               at org.opendaylight.yangtools.yang.data.impl.codec.SchemaTracker.startListItem(SchemaTracker.java:144)

               at org.opendaylight.yangtools.yang.data.codec.gson.JSONNormalizedNodeStreamWriter.startMapEntryNode(JSONNormalizedNodeStreamWriter.java:156)

               at org.opendaylight.yangtools.binding.data.codec.impl.BindingToNormalizedStreamWriter.startMapEntryNode(BindingToNormalizedStreamWriter.java:175)

               at org.opendaylight.yang.gen.v1.urn.opendaylight.netvirt.elan.rev150602.elan.instances.ElanInstance$StreamWriter.serialize(DataObjectSerializerPrototype.java)

               at org.opendaylight.yangtools.binding.data.codec.impl.BindingNormalizedNodeCodecRegistry$DataObjectSerializerProxy.serialize(BindingNormalizedNodeCodecRegistry.java:295)

               at org.opendaylight.newfederation.transform.TTPUtils.jsonStringFromDataObject(TTPUtils.java:146)

               at org.opendaylight.newfederation.tests.JsonParsing.main(JsonParsing.java:73)

 

Then, I tried to serialize the ElanInstances container initialized with the ElanInstance, and it succeeded, but with a lot of verbose code, that I guess could be reduced somehow.

This is the code:

SERIALIZER CODE:

        TTPUtils ttp = new TTPUtils(Collections.singleton(BindingReflections.getModuleInfo(ElanInstances.class)));

        ElanInstance newInstance = new ElanInstanceBuilder().setElanInstanceName("GUYELAN")

                .setDescription("ElanDescription").build();

        ArrayList<ElanInstance> list = new ArrayList<>();

        list.add(newInstance);

        ElanInstances instances = new ElanInstancesBuilder().setElanInstance(list).build();

        InstanceIdentifier<ElanInstances> id = InstanceIdentifier.create(ElanInstances.class);

 

        String jsonStringFromDataObject = ttp.jsonStringFromDataObject(id, instances, true);

        System.out.println(jsonStringFromDataObject);

DESERIALIZER CODE:

        NormalizedNode<?, ?> normalized = ttp.normalizedNodeFromJsonString(jsonStringFromDataObject);

        DataObject finalDataObject = ttp.dataObjectFromNormalizedNode("urn:opendaylight:netvirt:elan", "2015-06-02",

                "elan-instances", normalized);

        ElanInstances finalInstances = (ElanInstances) finalDataObject;

        System.out.println("RESULT: " + finalInstances.getElanInstance().get(0).getElanInstanceName());

 

 

I implemented the ttp.dataObjectFromNormalizedNode myself, like this:

public DataObject dataObjectFromNormalizedNode(String namespace, String revision, String localName, NormalizedNode<?, ?> nn) {

        QName qname = QName.create(namespace, revision, localName);

        return codecRegistry.fromNormalizedNode(YangInstanceIdentifier.of(qname), nn).getValue();

    }

 

Now to my questions:

1)      Is it possible to do what I’m trying to do with less “knowledge” in the deserializer side?  The deserializer had to know that the NormalizedNode it got was specifically belongs to this model:  ("urn:opendaylight:netvirt:elan", "2015-06-02", "elan-instances"), in order to do the deserialization.

2)      How will this work with augmentations that are declared in different models?

3)      Can I use it with a specific ElanInstance? (I got an exception as I illustrated).

4)      I have a feeling I am not doing this in the right way, is there any example of how to implement what I’m trying to do? If not, can someone help me with a code snippet ?

 

Thanks,

Guy Sela

 


_______________________________________________
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev