CXF and possible performance optimizations

[Jesper Duelund Isaksen]

Hello!

I am currently on a project where we are building a platform exposing a large number of standard WSDL-first SOAP services with typed SEI classes. The purpose of the platform is to do authentication and authorization for a large number of users before the users access external data systems. We are aware that it is not proper usage of SOAP services however we do not have control over our users.

We are however experiencing unexpected large size requests and responses to and from the platform (10-60 MB SOAP messages) and we are seeing that:


* It seems that all these large requests take up a large amount of memory since they are unmarshalled into Java object hierarchies and marshalled back to XML when the requests and responses are forwarded.
* A large amount of CPU time is used for garbage collecting of all these objects.

Additionally we are using the WS-SecurityPolicy functionality handled by Apache CXF since we in a WS-Trust-like setup for most of the services.

So the question is, is there any way we can avoid generating the Java object hierarchies and keep the using CXF for handling security?

Kind regards
Jesper

[Markus Schulz]

Am Freitag, 2. November 2018, 06:35:09 CET schrieb Jesper Duelund

Post by Jesper Duelund Isaksen
Hello!
I am currently on a project where we are building a platform exposing
a large number of standard WSDL-first SOAP services with typed SEI
classes. The purpose of the platform is to do authentication and
authorization for a large number of users before the users access
external data systems. We are aware that it is not proper usage of
SOAP services however we do not have control over our users.
We are however experiencing unexpected large size requests and
responses to and from the platform (10-60 MB SOAP messages) and we
* It seems that all these large requests take up a large amount of
memory since they are unmarshalled into Java object hierarchies and
marshalled back to XML when the requests and responses are forwarded.
* A large amount of CPU time is used for garbage collecting of all
these objects.
Additionally we are using the WS-SecurityPolicy functionality handled
by Apache CXF since we in a WS-Trust-like setup for most of the
services.
So the question is, is there any way we can avoid generating the Java
object hierarchies and keep the using CXF for handling security?

you can switch to dynamic SEIs with a provider based implementations.
take a look at http://cxf.apache.org/docs/provider-services.html.

you can still unmarshal on demand the request xml into a java object
structure.

regards
msc

[Jesper Duelund Isaksen]

Thanks for your suggestion Markus!

It looks like a WebServiceProvider for either the SAXSource or StreamSource would fit our needs.

However the note regarding developers needing to manually handle SOAP envelopes leads me to assume that everything WS-SecurityPolicy related needs to be manually handled as well. Would you consider that correct?

Kind regards
Jesper

-----Original Message-----
From: Markus Schulz [mailto:***@onesty-tech.de]
Sent: 2. november 2018 09:43
To: ***@cxf.apache.org
Subject: Re: CXF and possible performance optimizations

Am Freitag, 2. November 2018, 06:35:09 CET schrieb Jesper Duelund

Post by Jesper Duelund Isaksen
Hello!
I am currently on a project where we are building a platform exposing
a large number of standard WSDL-first SOAP services with typed SEI
classes. The purpose of the platform is to do authentication and
authorization for a large number of users before the users access
external data systems. We are aware that it is not proper usage of
SOAP services however we do not have control over our users.
We are however experiencing unexpected large size requests and
responses to and from the platform (10-60 MB SOAP messages) and we are
* It seems that all these large requests take up a large amount of
memory since they are unmarshalled into Java object hierarchies and
marshalled back to XML when the requests and responses are forwarded.
* A large amount of CPU time is used for garbage collecting of all
these objects.
Additionally we are using the WS-SecurityPolicy functionality handled
by Apache CXF since we in a WS-Trust-like setup for most of the
services.
So the question is, is there any way we can avoid generating the Java
object hierarchies and keep the using CXF for handling security?

you can switch to dynamic SEIs with a provider based implementations.
take a look at http://cxf.apache.org/docs/provider-services.html.

you can still unmarshal on demand the request xml into a java object structure.

regards
msc

[Markus Schulz]

Am Freitag, 2. November 2018, 09:08:34 CET schrieb Jesper Duelund

Post by Jesper Duelund Isaksen
Thanks for your suggestion Markus!
It looks like a WebServiceProvider for either the SAXSource or
StreamSource would fit our needs.
However the note regarding developers needing to manually handle SOAP
envelopes leads me to assume that everything WS-SecurityPolicy
related needs to be manually handled as well. Would you consider that
correct?

no, you don't need to handle security manually.
Full WS-SecurityPolicy (or other cxf-specific interceptors) are
available.
Simply put the ws-security-policy into your wsdl and let cxf do the hard
work for you.

@ServiceMode(value = Service.Mode.PAYLOAD)
@WebServiceProvider(wsdlLocation = "your.wsdl", ...)
public class MyProvider implements Provider<Source> {

@Override
public Source invoke(Source source) {
...unmarshal source on demand with jaxb or similar...
}

}

regards
msc

[Jesper Duelund Isaksen]

Alright, sounds great. I will have a deeper look into it.

Thanks for the quick response.

Kind regards
Jesper


-----Original Message-----
From: Markus Schulz [mailto:***@onesty-tech.de]
Sent: 2. november 2018 10:32
To: ***@cxf.apache.org
Subject: Re: CXF and possible performance optimizations

Am Freitag, 2. November 2018, 09:08:34 CET schrieb Jesper Duelund

Post by Jesper Duelund Isaksen
Thanks for your suggestion Markus!
It looks like a WebServiceProvider for either the SAXSource or
StreamSource would fit our needs.
However the note regarding developers needing to manually handle SOAP
envelopes leads me to assume that everything WS-SecurityPolicy related
needs to be manually handled as well. Would you consider that correct?

no, you don't need to handle security manually.
Full WS-SecurityPolicy (or other cxf-specific interceptors) are available.
Simply put the ws-security-policy into your wsdl and let cxf do the hard work for you.

@ServiceMode(value = Service.Mode.PAYLOAD) @WebServiceProvider(wsdlLocation = "your.wsdl", ...) public class MyProvider implements Provider<Source> {

@Override
public Source invoke(Source source) {
...unmarshal source on demand with jaxb or similar...
}

}

regards
msc

[Colm O hEigeartaigh]

CXF supports a streaming WS-Security implementation that might suit your
needs as well. You can enable it by setting the JAX-RS property
"ws-security.enable.streaming" to "true".

Colm.

On Fri, Nov 2, 2018 at 8:24 PM Jesper Duelund Isaksen <

Post by Jesper Duelund Isaksen
Alright, sounds great. I will have a deeper look into it.
Thanks for the quick response.
Kind regards
Jesper
-----Original Message-----
Sent: 2. november 2018 10:32
Subject: Re: CXF and possible performance optimizations
Am Freitag, 2. November 2018, 09:08:34 CET schrieb Jesper Duelund

Post by Jesper Duelund Isaksen
Thanks for your suggestion Markus!
It looks like a WebServiceProvider for either the SAXSource or
StreamSource would fit our needs.
However the note regarding developers needing to manually handle SOAP
envelopes leads me to assume that everything WS-SecurityPolicy related
needs to be manually handled as well. Would you consider that correct?

no, you don't need to handle security manually.
Full WS-SecurityPolicy (or other cxf-specific interceptors) are available.
Simply put the ws-security-policy into your wsdl and let cxf do the hard work for you.
@ServiceMode(value = Service.Mode.PAYLOAD)
@WebServiceProvider(wsdlLocation = "your.wsdl", ...) public class
MyProvider implements Provider<Source> {
@Override
public Source invoke(Source source) {
...unmarshal source on demand with jaxb or similar...
}
}
regards
msc

[Jesper Duelund Isaksen]

Thanks Colm. That is a very interesting option indeed even though it seems we have to loosen the Layout specification of the WS-Policy used.

Kind regards
Jesper

-----Original Message-----
From: Colm O hEigeartaigh [mailto:***@apache.org]
Sent: 5. november 2018 12:53
To: ***@cxf.apache.org
Subject: Re: CXF and possible performance optimizations

CXF supports a streaming WS-Security implementation that might suit your needs as well. You can enable it by setting the JAX-RS property "ws-security.enable.streaming" to "true".

Colm.

Post by Jesper Duelund Isaksen
Alright, sounds great. I will have a deeper look into it.
Thanks for the quick response.
Kind regards
Jesper
-----Original Message-----
Sent: 2. november 2018 10:32
Subject: Re: CXF and possible performance optimizations
Am Freitag, 2. November 2018, 09:08:34 CET schrieb Jesper Duelund

Post by Jesper Duelund Isaksen
Thanks for your suggestion Markus!
It looks like a WebServiceProvider for either the SAXSource or
StreamSource would fit our needs.
However the note regarding developers needing to manually handle
SOAP envelopes leads me to assume that everything WS-SecurityPolicy
related needs to be manually handled as well. Would you consider that correct?

no, you don't need to handle security manually.
Full WS-SecurityPolicy (or other cxf-specific interceptors) are available.
Simply put the ws-security-policy into your wsdl and let cxf do the hard work for you.
@ServiceMode(value = Service.Mode.PAYLOAD)
@WebServiceProvider(wsdlLocation = "your.wsdl", ...) public class
MyProvider implements Provider<Source> {
@Override
public Source invoke(Source source) {
...unmarshal source on demand with jaxb or similar...
}
}
regards
msc