Bug 2703 - Need to genererate ACE_HAS_EXPLICIT_TEMPLATE_INSTANTIATION_EXPORT
Summary: Need to genererate ACE_HAS_EXPLICIT_TEMPLATE_INSTANTIATION_EXPORT
Status: ASSIGNED
Alias: None
Product: TAO
Classification: Unclassified
Component: IDL Compiler (show other bugs)
Version: 1.5.3
Hardware: All All
: P3 normal
Assignee: Adam Mitz
URL:
Depends on:
Blocks:
 
Reported: 2006-11-10 03:30 CST by Johnny Willemsen
Modified: 2013-08-14 12:35 CDT (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Johnny Willemsen 2006-11-10 03:30:54 CST
We have not removed all generated file from the repo yet. We have in
tao/OctetSeqC.h the following code below. I am thinking about adding a new
option to tao_idl to indicate the template instantiation export has to be
generated for this file. At the moment the option is given in the header file
for the sequences in that file (at this moment I have only seen the problem with
the sequences) the IDL compiler generates for each sequence the code below.
Jeff, how much work would it be to have a special visitor to generate this code
in the header file?

// Workaround for a Visual Studio .NET bug where this class is not
// properly imported by an application if typedef'd or subclassed,
// resulting in 'multiply defined' link errors. The export macro
// here forces an explicit import by the application. Please see
// http://support.microsoft.com/default.aspx?scid=kb;en-us;309801
// The problem stems from use of the type below in PortableServer,
// but we put the instantiation here because the application will
// need to see it in *C.h to avoid the error.
#if defined ACE_HAS_EXPLICIT_TEMPLATE_INSTANTIATION_EXPORT
  template class TAO_Export TAO::unbounded_value_sequence<CORBA::OctetSeq>;
#endif /* ACE_HAS_EXPLICIT_TEMPLATE_INSTANTIATION_EXPORT */
Comment 1 Jeff Parsons 2006-11-10 13:37:51 CST
Do we need a separate visitor for this? Since we are going to be using this

(1) for a single file
(2) for the single sequence declaration in that file
(3) only until Microsoft fixes it

why don't we just add the option, which will generate the workaround for each 
sequence declaration in the file?
Comment 2 Jeff Parsons 2006-11-10 16:35:07 CST
I just noticed that the template parameter in the hand-written export in 
OctetSeqC.h is CORBA::OctetSeq. Shouldn't it be CORBA::Octet?

I also noticed (belatedly) that the export is generate at global scope, hence 
the reason for Johnny's suggestion that it be a separate visitor.
Comment 3 Johnny Willemsen 2006-11-12 07:34:05 CST
yes, should be without the seq. Besides octet this is also for ushort there and
the best fix would be to do this for all basic sequences. This is needed for vc8
and intel c++ 9.x at least, the msdn documentation says this must be done, with
intel c++ 9.1 we have seen problems linking user apps when they also have
typedefs for the basic sequences themselves
Comment 4 Johnny Willemsen 2006-11-22 14:41:14 CST
Wed Nov 22 20:18:50 UTC 2006  Jeff Parsons <j.parsons@vanderbilt.edu>
Comment 5 Adam Mitz 2007-02-13 15:31:08 CST
Hi Johnny and Jeff,

I have a few questions about this bug and -Gse in general.

1. Why do we need the #ifdef ACE_HAS_EXPLICIT_TEMPLATE_INSTANTIATION_EXPORT?
Shouldn't a forward-declaration be safe on any platform?

2. The support for "finding" typedefs via visitor seems incomplete.  Running
some test IDL through it shows that it will miss typedefs within structs and
valuetypes.  Also, by browsing the code it seems to miss typedefs within unions,
interfaces, and exceptions too.

3. The export macro is the current export and not TAO_Export.  The kernel of the
vc71/vc8 problem is that the client code doesn't recognize the sequence
specialization as an import from TAO.dll.  See the very first post in this
bugzilla for an example.  tao_idl currently uses the -Wb,export_macro setting
instead of TAO_Export, which makes it a dllexport instead of a dllimport from
the application's point of view.

Thanks,
Adam
Comment 6 Johnny Willemsen 2007-02-14 00:56:21 CST
First, this option is just for the core libraries, not for user applications

1. Why do we need the #ifdef ACE_HAS_EXPLICIT_TEMPLATE_INSTANTIATION_EXPORT?
Shouldn't a forward-declaration be safe on any platform?

* Nope, without the export you get duplicate link errors with msvc 71/8 when
using multiple libraries which use the same seqeuences


2. The support for "finding" typedefs via visitor seems incomplete.  Running
some test IDL through it shows that it will miss typedefs within structs and
valuetypes.  Also, by browsing the code it seems to miss typedefs within unions,
interfaces, and exceptions too.

* It is not meant for all types, only for the basic sequences. If you need it
for more types, feel free to extend the support

3. The export macro is the current export and not TAO_Export.  The kernel of the
vc71/vc8 problem is that the client code doesn't recognize the sequence
specialization as an import from TAO.dll.  See the very first post in this
bugzilla for an example.  tao_idl currently uses the -Wb,export_macro setting
instead of TAO_Export, which makes it a dllexport instead of a dllimport from
the application's point of view.

* We only use it in the core TAO lib, if you need it more, feel free to address
this.

Adam, I am reassigning this to you, if you think the support must be extended
feel free to do this
Comment 7 Adam Mitz 2007-02-14 10:20:28 CST
I should have explained the motivation behind my questions a little better.
We have seen a few cases where this does impact user's applications -- where the
user IDL has its own typedef for basic-type sequence.  For one example, see the
tao-users thread started by Birgit Platt (most recent message Feb 13 '07 8AM
UTC).  I didn't know the -Gse option existed until I was looking through the
voluminous tao_idl -u output.
It seems to me that -Gse can be made to fix the problem that Birgit Platt and
others have seen without requiring changes to the user's IDL.  My hypothesis is
that the functionality can be made general enough to not require a command-line
option or a #ifdef guard.

To respond to the individual points from the prior message...
(intentionally re-ordered)

2. I agree that it only applies to typedefs of sequences of basic types (those
that are already exported from TAO.dll via LongSeqC.h, etc).  My point here is
that the user's typedefs of sequences of basic types may be scoped within a
struct, interface, exception, union, or valuetype.  Actually I don't know that
all of these are legal in the IDL spec but at least some of them are.

3. This is the intent, so that whenever the IDL compiler emits a template
specialization that it knows is exported from TAO.dll it also emits a
forward-declaration with TAO_Export.

1. Given #3, I think #1 falls into place.  Since the export macro is TAO_Export
the user code should see a dllimport and link correctly.

Accepting bug.  I'll work on a patch for this -- let me know if you have
additional comments.
Comment 8 Johnny Willemsen 2007-02-14 12:46:01 CST
My response with a *

I should have explained the motivation behind my questions a little better.
We have seen a few cases where this does impact user's applications -- where the
user IDL has its own typedef for basic-type sequence.  For one example, see the
tao-users thread started by Birgit Platt (most recent message Feb 13 '07 8AM
UTC).  I didn't know the -Gse option existed until I was looking through the
voluminous tao_idl -u output.
It seems to me that -Gse can be made to fix the problem that Birgit Platt and
others have seen without requiring changes to the user's IDL.  My hypothesis is
that the functionality can be made general enough to not require a command-line
option or a #ifdef guard.

* JW, I don't think this is possible, only one library can export the base
template, if you do it automatically then it can happen that multiple libraries
export the base template and resolves in duplicate template instantiations. 

To respond to the individual points from the prior message...
(intentionally re-ordered)

2. I agree that it only applies to typedefs of sequences of basic types (those
that are already exported from TAO.dll via LongSeqC.h, etc).  My point here is
that the user's typedefs of sequences of basic types may be scoped within a
struct, interface, exception, union, or valuetype.  Actually I don't know that
all of these are legal in the IDL spec but at least some of them are.

* JW, yes, that can happen, but users can workaround easily around this, btw,
note that this is only for msvs71, msvs8 and intel c++. Other compilers don't
have this behaviour

3. This is the intent, so that whenever the IDL compiler emits a template
specialization that it knows is exported from TAO.dll it also emits a
forward-declaration with TAO_Export.

* JW, sounds interesting, but not sure if this will work

1. Given #3, I think #1 falls into place.  Since the export macro is TAO_Export
the user code should see a dllimport and link correctly.

* JW, ok, see your points, maybe this will work

Accepting bug.  I'll work on a patch for this -- let me know if you have
additional comments.

Comment 9 Jeff Parsons 2009-08-25 13:44:39 CDT
Johnny asked me in an email if it's possible for the IDL compiler to detect that a sequence of basic type is generated. The answer is yes, there are actually already many boolean flags that can be set if this or that type is seen in the IDL file - they are used now to be smart about generated includes. Both octet and string sequences are noted already if they are seen. Just check the boolean value idl_global->octet_seq_seen_ or idl_global->string_seq_seen_ (no accessors since there are so many of these flags, it would be cumbersome to have setters and getters for each one. Note that these flags are set only if the type in question is seen in the IDL file being processed, not for included types. If that is needed, a parallel mechanism will need to be put in place, but it wouldn't be hard to do.
Comment 10 Johnny Willemsen 2009-11-20 08:08:10 CST
the real solution is that the code we now generate into StringSeqC.h (and other SeqC.h) is generated at the top of the user generated C.h that uses a sequence of a basic type.
Comment 11 Adam Mitz 2009-11-20 16:29:44 CST
I'm not following exactly what problem we are trying to solve at this point.  Can you show an example of what's not working?  Alternatively, I'd be fine with this bug being assigned to someone else (I'd still like to be on the CC list).
Comment 12 Johnny Willemsen 2009-11-21 01:43:08 CST
when an end user doesn't include the stringseq.pidl file it also should work. Simon triggered this discussion through some emails again
Comment 13 davidh 2011-04-08 15:10:37 CDT
Adam asked me to follow-up with further comments (from a query to tao-users) here. Adam references the FAQ: http://www.theaceorb.com/faq/index.html#VCBasicSequenceLinkError, which states that either of 1) include orb.idl, *or* 2) use (e.g.) typedef CORBA::StringSeq MyType. However, both are needed in my case. Am I missing something here?
Thanks, -David
PS: I'm interested in this as a user only, and, as such, I don't claim to have any detailed understanding of the core issue(s) within TAO surrounding this bug.
Comment 14 Adam Mitz 2011-04-08 16:00:31 CDT
(In reply to comment #13)
> Adam asked me to follow-up with further comments (from a query to tao-users)
> here. Adam references the FAQ:
> http://www.theaceorb.com/faq/index.html#VCBasicSequenceLinkError, which states
> that either of 1) include orb.idl, *or* 2) use (e.g.) typedef CORBA::StringSeq
> MyType. However, both are needed in my case. Am I missing something here?

I don't think both are actually needed (#2 is not), but I agree that the FAQ isn't accurate (since #1 is).  I guess this must have changed in the few years since the FAQ was written -- see the circa 2009 comments on this bug.  Also, you can #include StringSeq.pidl instead of orb.idl if you want to reduce the work for the preprocessor and idl compiler.
I will make a note to update the FAQ when I have time.  There is also a related OpenDDS FAQ entry which has some more detail about using the pre-defined sequences (and doesn't have the inaccuracy you pointed out): http://www.opendds.org/faq.html#IDL_builtin_sequence


> PS: I'm interested in this as a user only, and, as such, I don't claim to have
> any detailed understanding of the core issue(s) within TAO surrounding this
> bug.

Understood.  I think it's valuable to have feedback from users in the bugzilla entries.
Comment 15 davidh 2011-04-08 17:32:25 CDT
(In reply to comment #14)
> (In reply to comment #13)
> > Adam asked me to follow-up with further comments (from a query to tao-users)
> > here. Adam references the FAQ:
> > http://www.theaceorb.com/faq/index.html#VCBasicSequenceLinkError, which states
> > that either of 1) include orb.idl, *or* 2) use (e.g.) typedef CORBA::StringSeq
> > MyType. However, both are needed in my case. Am I missing something here?
> 
> I don't think both are actually needed (#2 is not), but I agree that the FAQ
> isn't accurate (since #1 is).  I guess this must have changed in the few years
> since the FAQ was written -- see the circa 2009 comments on this bug.  Also,
> you can #include StringSeq.pidl instead of orb.idl if you want to reduce the
> work for the preprocessor and idl compiler.
> I will make a note to update the FAQ when I have time.  There is also a related
> OpenDDS FAQ entry which has some more detail about using the pre-defined
> sequences (and doesn't have the inaccuracy you pointed out):
> http://www.opendds.org/faq.html#IDL_builtin_sequence

Thanks Adam, the DDS entry makes sense. 

BTW, we use our IDL files across different IDL compilers/ORBS (e.g., Java) - are PIDL files portable in this way? Also, forgive the naiive question but I'm assuming CORBA::StringSeq is a newly defined CORBA type (what CORBA version was this defined in - my H&V does not include this)? If defined in the standard, why does orb.idl need to be included?
Comment 16 davidh 2011-04-14 11:05:25 CDT
(In reply to comment #15)
> (In reply to comment #14)
> > (In reply to comment #13)
> > > Adam asked me to follow-up with further comments (from a query to tao-users)
> > > here. Adam references the FAQ:
> > > http://www.theaceorb.com/faq/index.html#VCBasicSequenceLinkError, which states
> > > that either of 1) include orb.idl, *or* 2) use (e.g.) typedef CORBA::StringSeq
> > > MyType. However, both are needed in my case. Am I missing something here?
> > 
> > I don't think both are actually needed (#2 is not), but I agree that the FAQ
> > isn't accurate (since #1 is).  I guess this must have changed in the few years
> > since the FAQ was written -- see the circa 2009 comments on this bug.  Also,
> > you can #include StringSeq.pidl instead of orb.idl if you want to reduce the
> > work for the preprocessor and idl compiler.
> > I will make a note to update the FAQ when I have time.  There is also a related
> > OpenDDS FAQ entry which has some more detail about using the pre-defined
> > sequences (and doesn't have the inaccuracy you pointed out):
> > http://www.opendds.org/faq.html#IDL_builtin_sequence
> 
> Thanks Adam, the DDS entry makes sense. 
> 
> BTW, we use our IDL files across different IDL compilers/ORBS (e.g., Java) -
> are PIDL files portable in this way? Also, forgive the naiive question but I'm
> assuming CORBA::StringSeq is a newly defined CORBA type (what CORBA version was
> this defined in - my H&V does not include this)? If defined in the standard,
> why does orb.idl need to be included?

Just following up on this again. From a user's perspective these workarounds are problematic. For example, I've tried the Java 6 IDL compiler and it complains on both the include of orb.idl and, independently, the definition of CORBA::StringSeq. As a result the required workaround for this bug in the TAO IDL compiler results in non-portable IDL. Am I missing something here?
Comment 17 Adam Mitz 2011-04-14 11:09:52 CDT
> > 
> > BTW, we use our IDL files across different IDL compilers/ORBS (e.g., Java) -
> > are PIDL files portable in this way? Also, forgive the naiive question but I'm
> > assuming CORBA::StringSeq is a newly defined CORBA type (what CORBA version was
> > this defined in - my H&V does not include this)? If defined in the standard,
> > why does orb.idl need to be included?
> 
> Just following up on this again. 

Sorry I have not replied but I also have not forgotten about it.  This is on my radar but a lot of other project are too...

> From a user's perspective these workarounds
> are problematic. For example, I've tried the Java 6 IDL compiler and it
> complains on both the include of orb.idl and, independently, the definition of
> CORBA::StringSeq. As a result the required workaround for this bug in the TAO
> IDL compiler results in non-portable IDL. Am I missing something here?

Use the preprocessor:
#ifdef __TAO_IDL
...
#endif
Comment 18 davidh 2011-04-14 11:19:29 CDT
(In reply to comment #17)
> > > 
> > > BTW, we use our IDL files across different IDL compilers/ORBS (e.g., Java) -
> > > are PIDL files portable in this way? Also, forgive the naiive question but I'm
> > > assuming CORBA::StringSeq is a newly defined CORBA type (what CORBA version was
> > > this defined in - my H&V does not include this)? If defined in the standard,
> > > why does orb.idl need to be included?
> > 
> > Just following up on this again. 
> 
> Sorry I have not replied but I also have not forgotten about it.  This is on my
> radar but a lot of other project are too...
> 
> > From a user's perspective these workarounds
> > are problematic. For example, I've tried the Java 6 IDL compiler and it
> > complains on both the include of orb.idl and, independently, the definition of
> > CORBA::StringSeq. As a result the required workaround for this bug in the TAO
> > IDL compiler results in non-portable IDL. Am I missing something here?
> 
> Use the preprocessor:
> #ifdef __TAO_IDL
> ...
> #endif

;) Well, yes, of course, I can do this. But is this really a solution TAO wants to promote? This doesn't look very good when sharing IDL amongst vendors. Moreover it's also cumbersome to change all "typedef sequence<string> ..." references to include the following sequence:

#ifdef __TAO_IDL
#include "ord.idl"
#endif

#ifdef __TAO_IDL
typedef CORBA::StringSeq MyStringSeq;
#else
typedef sequence<string> MyStringSeq;
#endif

I regret that I'm unable to comment on the necessary IDL compiler modifications to fix this bug, but of all the ACE/TAO bugs solved over the last couple decades (wow - it's really been that long :)!) it seems like this one should be solvable as well?
Comment 19 Johnny Willemsen 2011-04-14 12:07:32 CDT
(In reply to comment #18)
> I regret that I'm unable to comment on the necessary IDL compiler modifications
> to fix this bug, but of all the ACE/TAO bugs solved over the last couple
> decades (wow - it's really been that long :)!) it seems like this one should be
> solvable as well?

This issue got caused by changed in the Microsoft compiler. As you can read in this bugzilla issue there are ideas to handle this better, but the challenge is time/funding, making this IDL compiler changes takes time and funding for resolving this issue is welcome

Comment 20 davidh 2011-04-14 14:39:58 CDT
(In reply to comment #19)
> (In reply to comment #18)
> > I regret that I'm unable to comment on the necessary IDL compiler modifications
> > to fix this bug, but of all the ACE/TAO bugs solved over the last couple
> > decades (wow - it's really been that long :)!) it seems like this one should be
> > solvable as well?
> 
> This issue got caused by changed in the Microsoft compiler. As you can read in
> this bugzilla issue there are ideas to handle this better, but the challenge is
> time/funding, making this IDL compiler changes takes time and funding for
> resolving this issue is welcome

Yes, I understand the need to evaluate changes with respect to funding or not. Given that this is happening in Microsoft's latest compiler (VS2010 SP1), and has been happening for some time in its compilers, I would have thought that this issue would have been addressed outright (i.e., without funding). However, I surely don't know the ins and outs of how the core ACE/TAO developers evaluate these criteria (i.e., which bugs are dealt with outright and which ones require additional funding). 

I have to say I'm also a bit surprised that this hasn't been raised more frequently in the user community (I've seen just a few references to this) over the past five years. Either: 1) folks are just OK with this limitation, 2) folks don't tend to share their IDL files between ORB vendors, and/or 3) folks don't tend to re-define (typdef) "standard" sequences.

BTW, I'm still not clear on whether the CORBA::StringSequence et al types are standard or not. Are they? If so, why is orb.idl needed? If not, how is it that these types are defined in the CORBA namespace?

Comment 21 Johnny Willemsen 2011-04-15 03:01:57 CDT
(In reply to comment #20)
> BTW, I'm still not clear on whether the CORBA::StringSequence et al types are
> standard or not. Are they? If so, why is orb.idl needed? If not, how is it that
> these types are defined in the CORBA namespace?

They are standard as far as I know, but no time at this moment to start searching the specs to find it 

Comment 22 davidh 2011-04-15 15:26:15 CDT
(In reply to comment #21)
> (In reply to comment #20)
> > BTW, I'm still not clear on whether the CORBA::StringSequence et al types are
> > standard or not. Are they? If so, why is orb.idl needed? If not, how is it that
> > these types are defined in the CORBA namespace?
> 
> They are standard as far as I know, but no time at this moment to start
> searching the specs to find it 

Then why is '#include "orb.idl"' required (this isn't needed for other pre-defined types)?

Comment 23 Johnny Willemsen 2013-08-14 12:35:57 CDT
Wed Aug 14 17:34:10 UTC 2013  Johnny Willemsen  <jwillemsen@remedy.nl>

        * tao/corba.h:
          Document that this file is for backward compatibility
          only. Thanks to Lukas Grützmacher <Lukas dot Gruetzmacher
          at ais-automation dot com> that this was not done.