Bug 2529

On Tue, 21 Feb 2006 05:01:38 -0600, bala@dre.vanderbilt.edu (Balachandran 
Natarajan) said:

> Nanbor- Hopefully, I will send this out by today. Thanks for the
> long email and some thought provoking insights. Please see below.

> On Monday, 20 February, 2006 at 00:14:46 -0600, nanbor@txcorp.com
> via RT wrote:
>> in composing and sending out this email which I hope will
>> illustrate why I don't think adding the non-standard
>> ciao_preactivate and ciao_postactivate methods is the right
>> approach.
>> 
>> I think it's fair to say that the main motivation of adding
>> ciao_preactivate and ciao_postactivate methods is to provide a
>> mechanism for developers to enforce the order of initialization of
>> many components in an assembled application.  Below, I'll try to
>> argue that this is the wrong approach to solve this problem.
>> 
>> 1. These added methods are *non-standard* should require no further
>> saying.  Moreover, they only help address a limited cases of
>> "component initialization order" problem.  In another word, they
>> were added to address a specific problem that seemingly represent a
>> generic problem.  (I.e., it's more of a hack than a general
>> solution.)

>        Okay. Let me accept this premise (as a hack) for now.

I think my last sentence above probably came across in the wrong way.
Please don't take it personally and my sincere apology if I offend you
in anyway.  Don't forget that Ossama used to think that "Nanbor" was a
code name for hackers. ;-)

>> The problem that ORBInitializer pre_init and post_init methods try
>> to address is very specific, i.e., to support a certain new feature
>> in the ORB, something needs to be done before or after ORB
>> initialization.  These methods have a very definite reference time,
>> the ORB::init call.  Unlike the ORBInitializer methods, there is
>> really no fixed reference call for component activation.  There is
>> only a reference time for *a specific application".

>         Here is where I differ. You make it sound as if, a component
> is _an_ application all by itself. In the type of systems that CIAO
> is focusing on, a component is just _a_piece_ of the
> puzzle. 

Tout au contraire!  My argument is built on the fact that a
application consists multiple components.  Component model allows
developers to change the composition of an application easily.
Therefore, it's not appropriate to hard-code the application-specific
initialization logic/semantic inside a component.  My sincere apology
for not getting the point across.

> Components need to to work with other components to do anything
> useful. 

Absolutely.

> If you accept this, then a component instantiation has a very
> definite reference time (in your parlance).

Not quite.  To which component do you make it the "reference time?"
Precisely because there are potentially many components in an
"application", there's not a "main" initialization point like
ORB_init.  

>> Along with ccm_activation, the added ciao_preactivate and
>> ciao_postactivate calls provide developers 3 distinct levels of
>> ordered initialization.  I will argue that 3 is not generic enough.
>> In fact, I will argue that there should not be a preset number of
>> initialization levels.

>         This is not about initialization levels. This is about
> preset callbacks/hooks/control points that can be externalized and
> controlled when pieces of your application go live. The pre call
> allows you to set up resources that can be used as part of the
> regular activate call, and the post is for cleanup activities. You
> can see this paradigms in many use cases. POA's pre_invoke () and
> post_invoke () is the one probably the most easiest to connect (in
> addition to pre_init () and post_init ()). And you will see below
> why 3 is sufficient.

Again, the example you use, method invocation in POA, there is a clear
reference point.  It makes sense to do something before that and after
that.  

>> Take the ARMS use-case for example.  In the 3 levels of
>> initialization, we would do:

> As an aside ARMS would have been much simpler with just a pre*
> call. The post_* is redundant for ARMS. post* is nice from
> symmetricity perspective.

I agree that ARMS use-case is trivial but that's beside the point
here. 

>> (1) Getting the logging service ready to work properly.  I.e, after
>> pre-activate, we can use the logging service to "log" stuff.

> Okay. Just a little digresison here:

> In ARMS the logging service is _actually_ a component. It abstracts
> away some details of the underlying logger. The logger required that
> an init () (or an equivalent) be called before use. This implied
> that the logger component needs to be intialized first. 

Right.  I knew logging service is a component and why the preactivate
call is used to hard code the fact that logging service is used as a
service for other components.  What if someday, you want to initialize
another service that logging service depends on?  You will need a
prepreactivate call to do that.

> CCM doesn't provide a way to order component initializations based
> on dependencies. 

I agree with this. :)

> Hence the pre* was done to provide some guarantee to the application

But my point is that using pre/post-activate calls is not a general
solution.  

> And, oh, BTW this (ordering/dependency between components) is a very
> very common problem. 

I agree with you on this statement completely.

> I saw it in ARMS. I see it in the software I am writing. 

I am sure they have the same order of initialization problem also.

> The Bob program saw it too (the app strings used to have an ordering
> of software artifacts in the string definitions in those
> days). 

Not sure what Bob program you are referring to...

> Everyone seems to have their own way of solving it.

Absolutely, there are many ways to skin a cat, just like all computer
science or architecturing problems.  However, there are always
tradeoffs in making these decisions.

>> (2) Getting most of other component ready to work in ccm-activate.

> And in addition, provide a guarantee that other components (whether
> remote or collocated) has a valid reference (and is ready to serve
> requests).

I'm not sure what you mean here.  A component can certainly check if
they have the references they need to function properly.  

>>  (3) Getting the application ready to work by using the
>> post-activate methods for a "facade" component.

> I am not sure what you mean by a "facade" component.

Sorry, I was trying to justify the need for a post-activate call
useful in your example.  

> You seem to be talking about "one" component, and I am (and will be)
> worried about "componentS" that make up my application. Some
> componentS in the same process space can probably share the same
> component for say logging.

Again, contrary to your assumption, I'm thinking of applicatons with
multiple components.

>> Why would this scenario not enough for CCM?  Remember, CCM is all
>> about flexibility and dynamic composition of components.

> No question on the flexibility aspects of CCM. When trying to throw
> in requirements for DRE system (including the software that we are
> working on), the vanilla definition of component doesn't seem to
> hold. You not only need dynamic composition of components, but also
> need much better control of your environment in which these
> components are executing.

Not argument about the need.  The question is how do one achieve that
goal without sacrificing what CCM brings us.

>> What if we want to replace a different logging service that utilize
>> another foobar event-channel-like component to delivery the logs to
>> anyone interested in monitoring the system?  We would need to make
>> sure the foobar component is ready to work before the logging
>> component.  In this case, how do we make sure the foobar component
>> gets initialized before the logging component in the application?
>> Do we modify CIAO to add the ciao_prepreactivate method?
>> Similarly, one can imagine the need to have a ciao_postpostactivate
>> method can come up someday...

>         IMHO, you are missing the point. pre* calls only provides a
> guarantee that they are called before ccm_activate is called. It
> doesn't provide a global ordering guarantee.

Uh, that's exactly my point.  pre* calls alongs don't provide enough
ordering in component initialization.

> More reason why 3 levels are sufficient. 

But I see that it just shows that 3 levels are not enough.  Since you
can always need the 4th and the 5th level as an application evolves.  

> Many applications that I am aware off needs to ensure that their
> dependencies (or resources or whatever) are up before they use
> them. And here is how your use case will pan out (please see how
> your requirement for prepre calls is a moot):

You mean a component using preactivate will never depend on another
component?  

> (1) ciao_preactivate() will be invoked on all components, i.e., on
> foobar, logging and the application component (ordering doesn't
> matter).

What if they matter for some components?

> (2) during ccm_activate () any of the components say foobar or
> logging can use each other . Why? There is a guarantee that the
> other component is up due to the contract the CIAO's middleware
> system provides.

> (3) during ciao_post* call, some intializations can be deferred if
> needed.

> I don't see a reason why this wouldn't work. And I can probably try
> establishing why this would be the most preferred way to do this
> :-)!

My example of logging service depending on another component (such as
a specialized event delivery service) to
perform the actual logging at a remote location will break since you
can not guarantee that it will be pre-activated before logging
service. 
        
>> SO, we can see, the ciao_preactivate and ciao_postactivate only
>> solved the order of initialization problem for the use-case that's
>> specific to the ARMS use-case.

> Me feels that you got it wrong without understanding the semantics.

I beg to defer.  I thought I had a pretty good understanding on how it
worked and what it tried to address.

>> 2. This one is the real killer.  By introducing the
>> ciao_preactivate and ciao_postactivate methods, we in effect are
>> pushing application specific logic, i.e., the order of
>> initialization in a specific application, into component
>> implications.

> I completely disagree. Me thinks that we provide an added
> flexibility by:

> (1) allowing developers to abstract out environment or deployment
> details to a specific component and

Abstract and hard-code the environment and deployment details _inside_
the component implementation.

> (2) providing a guarantee that the environment controller is up and
> running before the actual application that needs to be running in
> that environment is live.

Which also scatter among multiple components.  This is equivalent to
tying these components together and prevent them to be reused with other
components, under different environment.

> It all depends on how one wants to leverage such features in large
> scale embedded systems.

Or hard-code your components for one context.

>> This, in turn, reduce the reusability of a component
>> implementation.  For example, the logging service component
>> mentioned earlier can only be used in applications that use it as a
>> pre-initialized service.

> I think you have a uni-dimensional definition of a component, and
> you also assume that a component by virtue should be resuable.

I'm not sure which dimension you are referring to but yes, I do think
that a component should be reusable.  Do I miss something?

> I say not necessary. You write reusable classes and you also write
> helper classes. Your helper classes (that kitchen sink of a class)
> would be making reusable classes more effective. Similarly you write
> reusable components which embeds pieces of application logic and you
> abstract out the details not expressible in descriptors, into a
> software entity which can be hosted using the same infrastructure
> (and one that would make the reusable component very
> effective). Think of these as helper components that are deployment
> specific.

Components should be reusable.  Even for helper components that are
seemingly hard to reuse, one would eventually figure out some usage
patterns and try to generalize it so they can be used in a broader
categories of applications.  

>> As we can see, components utilizing these methods are in effect
>> restricting how they can be used in an application assembly that
>> requires the specific initialization order.

> Please see above.

>> The ccm_activate and ccm_passivate are really just methods the CCM
>> runtime system calls back to component implementations to get them
>> ready to begin or finish "normal functioning" by, for example,
>> acquiring and releasing resources or verifying a component
>> implementation are in a state for "normal functioning" (such as
>> having all the receptacle connected.)

> Sorry, it is not "just". I am very sorry for not getting this very
> (the intent) clearly across earlier.

>> The key here is that the component need to verify this by itself
>> without depending on other component.  This helps ensure the
>> reusability of a component implementation.  In our previous
>> example, component utilizing the loggging service can only be used
>> with the logging service with that specific initialization order.

> Please see above.

>> What might be a solution to address this "order of initialization"
>> problem?  I think we should explore how to apply the DnC spec to
>> achieve this goal and to avoid modifying the underlying CCM runtime
>> infrastructure.  One thing I'm sure is that the
>> order-of-initialization if an application-specific problem.
>> Therefore, it should be dealt with at the application-level, rather
>> than in the component-implementation-level.

> Couldn't have been better. Yes, order-of-initialization is
> application specific problem. Will your middleware make my task
> easier when I am dealing with 'n' number of dependencies?

Most definitely.  

>> One naive and straightforward way of handling this is to treat
>> components at different init-order as separate applications.  Using
>> this approach, an application assembly under the old approach will
>> then be split into multiple sub-applications that will be deployed
>> in certain order.  This will ensure the earlier deployed
>> sub-applications (e.g, the logging service sub-application) are
>> fully functional before other sub-applications are started.  A
>> simple script that invokes plan-launcher multiple times using
>> different "sub-application assemblies" is a simple way of achieving
>> this goal.

> Oh my! Have you been spending too much time with component folks?
> :-) 

Is that a surprise to you? :-p

> How can I handle components that need to be collocated in the same
> process space as the "application" component?

I think you can easily call out the same NodeApplication when you
install the second batch of the application.  I never said that CIAO
already support this, mind you. :)

>> Another, more elegant approach, is to enhance DnC spec to
>> explicitly define the order of initialization in a hierarchical
>> (nested) assembly.  Since DnC allows assemblies to be viewed as
>> components, if the deployment mechanism can ensure that the
>> sub-application assemblies will always be fully deployed before
>> upper level assemblies, the problem is easy to solve.  For the ARMS
>> use-case, the logging component can then be defined as a single
>> component assembly which will implicitly instruct the enhanced DnC
>> mechanism to finalize the component before deploying the others.
>> This approach is more involved and will likely need to modify the
>> DnC spec.

> Yup! Too complex and hard to get it right.

We haven't even tried or put too much thought into it yet.  :)  

>> application-level.  Anyhow, I hope the above analysis makes sense
>> to you folk.

> Not for me, since your analysis assumes some stereotyped definitions
> and usage rules. I don't think (and believe) CIAO will make a cut in
> those areas. 

I'm not sure what you mean by stereotyped definitions and usage
rules.  

> TAO is successful (and so will CIAO be) since it stepped away from
> canonical usage models and exposed what is important for
> applications in a proprietary manner.

I have no doubt that TAO addresses tough problems in ORB design and
thus is very successful.  I don't think that I'm suggesting that CIAO
should try to stay mediocre by avoiding hard problems.  I'm just
suggesting that there should be a way to address these hard problems
while staying true to the component programming model. 

>> If not, please let me know.

> Please see above.

Again, I'm sorry that I didn't get my point across the first time.  I
believe you're reply further validate my claim.  Hopefully, I am
making my point clearer now.  

Thanks,

nanbor