By: squarecog

squarecog — Sat, 16 Oct 2010 19:30:38 +0000

Hi David,
I saw your ticket, it’s certainly the next logical thing to do with invokers. They can probably be extended to handle using a cached object to call methods on, though the syntax might get a little funky unless we modify Pig Latin to accomodate doing that sort of thing easier. I’ll think about it, your request is a common one.

As far as performance overhead, writing a native Pig wrapper eval will be faster.

By: David Ciemiewicz

David Ciemiewicz — Sat, 16 Oct 2010 19:21:13 +0000

Hey, thanks or writing this capability.

Unfortunately, I need more than just static functions. For instance, I needed to compute the Incomplete Beta – P(x>value) given a Beta distribution with parameters of Beta(alpha, beta). And the Incomplete Beta computation parameters of x, alpha, and beta vary per record of computation.

The way the Apache Commons library is implemented, first I must create a BetaDistributionImpl(alpha, beta), then I must invoke the dynamic (non-static) method cumulativeProbability(x).

I submitted a JIRA proposal with some sketches of how I’d like this to work:

https://issues.apache.org/jira/browse/PIG-1678

I don’t know if the Invoker system can be easily extended to handle the case of first constructing the class object and then invoking the associated dynamic method, or not.

Also, regarding the performance overhead, is the overhead of the Invoker the same as writing a native Pig wrapper eval function? It wasn’t clear. Would making invoker’s first class implementations in Pig remove any of the overhead?

Comments on: Upcoming Features in Pig 0.8: Dynamic Invokers

By: squarecog

By: David Ciemiewicz