Please report new issues athttps://github.com/DOCGroup
I have seen an issue where the TAO_Singleton_Manager registers with the ACE object manager, at shutdown, the TAO dll is already unloaded and then the ACE object manager want todo a cleanup call to the TAO_Singleton_Manager which then results in a crash. In the code below, on line 206 we want to call the cleanup of the TAO_Singleton_Manager Program received signal SIGSEGV, Segmentation fault. 0x00002aaaae3ee470 in ?? () (gdb) list 172 173 return false; 174 } 175 176 void 177 ACE_OS_Exit_Info::call_hooks (void) 178 { 179 // Call all registered cleanup hooks, in reverse order of 180 // registration. 181 for (ACE_Cleanup_Info_Node *iter = registered_objects_; (gdb) up #1 0x00002aaaaade13ce in ACE_OS_Exit_Info::call_hooks (this=0x1273e298) at Cleanup.cpp:206 206 (*info.cleanup_hook_) (info.object_, info.param_); (gdb) list 201 // The hook is an ACE_EXIT_HOOK. 202 (* reinterpret_cast<ACE_EXIT_HOOK> (info.cleanup_hook_)) (); 203 } 204 else 205 { 206 (*info.cleanup_hook_) (info.object_, info.param_); 207 } 208 } 209 210 // Delete now the list of cleanup hooks, all are called and we don't want
Below the callstack when the tao singleton is registered. see that this is done through service config. Iliyan, any ideas on this? (gdb) back #0 ACE_OS_Exit_Info::at_exit_i (this=0x1f65f298, object=0x1f6de750, cleanup_hook=0x2aaaae3ee470 <TAO_SINGLETON_MANAGER_CLEANUP_DESTROYER_NAME>, param=0x0) at Cleanup.cpp:136 #1 0x00002aaaaae15683 in ACE_Object_Manager::at_exit_i (this=0x1f65f280, object=0x1f6de750, cleanup_hook=0x2aaaae3ee470 <TAO_SINGLETON_MANAGER_CLEANUP_DESTROYER_NAME>, param=0x0) at Object_Manager.cpp:465 #2 0x00002aaaae3ee0c0 in TAO_Singleton_Manager::init (this=0x1f6de750, register_with_object_manager=1) at /opt2/linux/x86_64/ACE/svn-latest/ACE_wrappers/ace/Object_Manager.inl:27 #3 0x00002aaaae3ead87 in TAO::ORB::open_global_services (argc=10, argv=0x1f6de4a0) at TAO_Internal.cpp:220 #4 0x00002aaaae3b69b9 in CORBA::ORB_init (argc=@0x7fff9a73a674, argv=0x1f6de4a0, orbid=0x0) at ORB.cpp:1257 #5 0x00002aaaacf6113d in X::utility::DllOrb::init (this=0x1f69af30, argc=10, argv=0x1f6de4a0) at src/corbautility/DllOrb.cpp:93 #6 0x00002aaaaadbfe6d in ACE_Service_Object_Type::init (this=0x1f680d20, argc=527296336, argv=0x2aaaae3ee470) at Service_Types.cpp:105 #7 0x00002aaaaadbaa9e in ACE_Service_Gestalt::initialize_i (this=0x1f65f6f0, sr=0x1f671ac0, parameters=<value optimized out>) at Service_Gestalt.cpp:655 #8 0x00002aaaaadbad2a in ACE_Service_Gestalt::initialize (this=0x1f65f6f0, sr=0x1f671ac0, parameters=0x1f67df70 "DllOrb -NumThreads 2 -ORBGestalt CURRENT -ORBDebugLevel 0 $TS_ORB_ENDPOINT -ORBDottedDecimalAddresses 1 -ORBCollocationStrategy thru_poa") at Service_Gestalt.cpp:643 #9 0x00002aaaac7dd671 in ACEXML_Svcconf_Handler::endElement (this=0x1f69b1b0, qName=<value optimized out>) at /opt2/linux/x86_64/ACE/svn-latest/ACE_wrappers/ace/Service_Config.inl:140 #10 0x00002aaaac9eb626 in ACEXML_Parser::parse_content (this=0x1f69b028, startname=0x1f69b2a1 "dynamic", ns_uri=@0x7fff9a73b4a8, ns_lname=@0x7fff9a73b4a0, ns_flag=0) at Parser.cpp:1072 #11 0x00002aaaac9eb0d3 in ACEXML_Parser::parse_element (this=0x1f69b028, is_root=<value optimized out>) at Parser.cpp:982 #12 0x00002aaaac9eb553 in ACEXML_Parser::parse_content (this=0x1f69b028, startname=0x1f69b294 "ACE_Svc_Conf", ns_uri=@0x7fff9a73b5e8, ns_lname=@0x7fff9a73b5e0, ns_flag=0) at Parser.cpp:1089 #13 0x00002aaaac9eb0d3 in ACEXML_Parser::parse_element (this=0x1f69b028, is_root=<value optimized out>) at Parser.cpp:982 #14 0x00002aaaac9f0666 in ACEXML_Parser::parse (this=0x1f69b028, input=<value optimized out>) at Parser.cpp:223 #15 0x00002aaaac7dad90 in ACEXML_Svcconf_Parser::parse_file (this=0x1f69b020, file=0x1f6725c8 "/home/johnny/test/X/config/clearingtool-start.xml") at Svcconf.cpp:71 #16 0x00002aaaaadbb368 in ACE_Service_Gestalt::process_file (this=<value optimized out>, file=0x1f6725c8 "/home/johnny/test/X/config/clearingtool-start.xml") at Service_Gestalt.cpp:951 #17 0x0000000000417cd1 in ACE_Service_Config::process_file (file=0x1f6725c8 "/home/johnny/test/X/config/clearingtool-start.xml") at /opt2/linux/x86_64/ACE/johnny/ACE_wrappers/ace/Service_Config.inl:147 #18 0x0000000000408bd1 in processConfigFile (r_serviceConfig=@0x7fff9a73c120, rc_filename=@0x7fff9a73cb00) at src/loader/main.cpp:453 #19 0x0000000000409413 in run (rc_applicationname=@0x7fff9a73cb10, rc_startfile=@0x7fff9a73cb00, rc_reconfigurefile=@0x7fff9a73caf0, rc_repairfile=@0x7fff9a73cd60) at src/loader/main.cpp:516 #20 0x00000000004119a5 in main (argc=6, t_argv=0x7fff9a73d008) at src/loader/main.cpp:832
(In reply to comment #1) > Below the callstack when the tao singleton is registered. > See that this is done through service config. > > Iliyan, any ideas on this? Johnny, your description indicates that the process is loading TAO dll and registering TAO_Singleton_Manager with ACE Object Manager. I suspected that at the same time, it doesn't up the ref count on the corresponding ACE_DLL object. However the stack trace below is peculiar in the fact that ACE_OS_Exit_Info::at_exit_i gets called _during_ TAO_Singleton_Manager::init. It almost seems like an automatic instance is being destroyed, and with it the DllOrb dll is being unloaded ... before it has been (fully) loaded?! My hunch would be to give it another try, using the non-xml configuration file. My suspicion is that the sequence of events is not quite the same as with the xml interpreter. > (gdb) back > #0 ACE_OS_Exit_Info::at_exit_i (this=0x1f65f298, object=0x1f6de750, > cleanup_hook=0x2aaaae3ee470 <TAO_SINGLETON_MANAGER_CLEANUP_DESTROYER_NAME>, > param=0x0) > at Cleanup.cpp:136 > #1 0x00002aaaaae15683 in ACE_Object_Manager::at_exit_i (this=0x1f65f280, > object=0x1f6de750, > cleanup_hook=0x2aaaae3ee470 <TAO_SINGLETON_MANAGER_CLEANUP_DESTROYER_NAME>, > param=0x0) at Object_Manager.cpp:465 > #2 0x00002aaaae3ee0c0 in TAO_Singleton_Manager::init (this=0x1f6de750, > register_with_object_manager=1) > at /opt2/linux/x86_64/ACE/svn-latest/ACE_wrappers/ace/Object_Manager.inl:27 > #3 0x00002aaaae3ead87 in TAO::ORB::open_global_services (argc=10, > argv=0x1f6de4a0) at TAO_Internal.cpp:220 > #4 0x00002aaaae3b69b9 in CORBA::ORB_init (argc=@0x7fff9a73a674, > argv=0x1f6de4a0, orbid=0x0) at ORB.cpp:1257 > #5 0x00002aaaacf6113d in X::utility::DllOrb::init (this=0x1f69af30, argc=10, > argv=0x1f6de4a0) at src/corbautility/DllOrb.cpp:93 > #6 0x00002aaaaadbfe6d in ACE_Service_Object_Type::init (this=0x1f680d20, > argc=527296336, argv=0x2aaaae3ee470) at Service_Types.cpp:105 > #7 0x00002aaaaadbaa9e in ACE_Service_Gestalt::initialize_i (this=0x1f65f6f0, > sr=0x1f671ac0, parameters=<value optimized out>) at Service_Gestalt.cpp:655 > #8 0x00002aaaaadbad2a in ACE_Service_Gestalt::initialize (this=0x1f65f6f0, > sr=0x1f671ac0, > parameters=0x1f67df70 "DllOrb -NumThreads 2 -ORBGestalt CURRENT > -ORBDebugLevel 0 $TS_ORB_ENDPOINT -ORBDottedDecimalAddresses 1 > -ORBCollocationStrategy thru_poa") at Service_Gestalt.cpp:643 > #9 0x00002aaaac7dd671 in ACEXML_Svcconf_Handler::endElement (this=0x1f69b1b0, > qName=<value optimized out>) > at > /opt2/linux/x86_64/ACE/svn-latest/ACE_wrappers/ace/Service_Config.inl:140 > #10 0x00002aaaac9eb626 in ACEXML_Parser::parse_content (this=0x1f69b028, > startname=0x1f69b2a1 "dynamic", ns_uri=@0x7fff9a73b4a8, > ns_lname=@0x7fff9a73b4a0, > ns_flag=0) at Parser.cpp:1072 > #11 0x00002aaaac9eb0d3 in ACEXML_Parser::parse_element (this=0x1f69b028, > is_root=<value optimized out>) at Parser.cpp:982 > #12 0x00002aaaac9eb553 in ACEXML_Parser::parse_content (this=0x1f69b028, > startname=0x1f69b294 "ACE_Svc_Conf", ns_uri=@0x7fff9a73b5e8, > ns_lname=@0x7fff9a73b5e0, ns_flag=0) at Parser.cpp:1089 > #13 0x00002aaaac9eb0d3 in ACEXML_Parser::parse_element (this=0x1f69b028, > is_root=<value optimized out>) at Parser.cpp:982 > #14 0x00002aaaac9f0666 in ACEXML_Parser::parse (this=0x1f69b028, input=<value > optimized out>) at Parser.cpp:223 > #15 0x00002aaaac7dad90 in ACEXML_Svcconf_Parser::parse_file (this=0x1f69b020, > file=0x1f6725c8 "/home/johnny/test/X/config/clearingtool-start.xml") > at Svcconf.cpp:71 > #16 0x00002aaaaadbb368 in ACE_Service_Gestalt::process_file (this=<value > optimized out>, > file=0x1f6725c8 "/home/johnny/test/X/config/clearingtool-start.xml") at > Service_Gestalt.cpp:951 > #17 0x0000000000417cd1 in ACE_Service_Config::process_file (file=0x1f6725c8 > "/home/johnny/test/X/config/clearingtool-start.xml") > at /opt2/linux/x86_64/ACE/johnny/ACE_wrappers/ace/Service_Config.inl:147 > #18 0x0000000000408bd1 in processConfigFile (r_serviceConfig=@0x7fff9a73c120, > rc_filename=@0x7fff9a73cb00) at src/loader/main.cpp:453 > #19 0x0000000000409413 in run (rc_applicationname=@0x7fff9a73cb10, > rc_startfile=@0x7fff9a73cb00, rc_reconfigurefile=@0x7fff9a73caf0, > rc_repairfile=@0x7fff9a73cd60) at src/loader/main.cpp:516 > #20 0x00000000004119a5 in main (argc=6, t_argv=0x7fff9a73d008) at > src/loader/main.cpp:832 >
> Johnny, your description indicates that the process is loading TAO dll and > registering TAO_Singleton_Manager with ACE Object Manager. I suspected that at > the same time, it doesn't up the ref count on the corresponding ACE_DLL object. > > However the stack trace below is peculiar in the fact that > ACE_OS_Exit_Info::at_exit_i gets called _during_ TAO_Singleton_Manager::init. This is pure the registration of the singleton manager with ACE, at_exit_i is just the name > My hunch would be to give it another try, using the non-xml configuration file. > My suspicion is that the sequence of events is not quite the same as with the > xml interpreter. Just also found that we where using -orbgestalt current, removed that, but doesn't solve the issue. seems the same issue with ssl, see the other bugzilla updates of today.
(In reply to comment #3) > > Johnny, your description indicates that the process is loading TAO dll and > > registering TAO_Singleton_Manager with ACE Object Manager. I suspected that at > > the same time, it doesn't up the ref count on the corresponding ACE_DLL object. > > > > However the stack trace below is peculiar in the fact that > > ACE_OS_Exit_Info::at_exit_i gets called _during_ TAO_Singleton_Manager::init. > > This is pure the registration of the singleton manager with ACE, at_exit_i is > just the name Without looking at the code at the moment I'm not sure I follow. Are you saying that at_exit_i() is being called for the name object/instance? > > My hunch would be to give it another try, using the non-xml > > configuration file. > > My suspicion is that the sequence of events is not quite the > > same as with the > > xml interpreter. > > Just also found that we where using -orbgestalt current, removed that, but > doesn't solve the issue. seems the same issue with ssl, see the other bugzilla > updates of today. The singletons are quite the finicky beasts, but I can't remember anything like this. Did you get a chance to try with the plain-text config file?
looking at the code it seems the tao_singleton_manager can register with the ace object manager, but will never unregister when it gets explicitly deleted. ace should deliver methods to unregister an at_exit hook
> Without looking at the code at the moment I'm not sure I follow. Are you saying > that at_exit_i() is being called for the name object/instance? It is just the registration, not cleanup
* tao/TAO_Singleton_Manager.cpp (fini): When we have been registered with the ACE object manager then unregister ourselves again. If we keep registered and the TAO DLL gets unloaded ACE will crash when trying to callback to us