Discussion:
COM+ Applications hanging
(too old to reply)
Eric Margheim
2005-10-21 18:34:55 UTC
Permalink
I have two ATL Dlls that have multiple interfaces and run under two COM+
Applications. Each DLL accesses SQL Server db via ADO calls and one of
the components also interacts with a 3rd party report server system.

We run into intermittent issues with the COM+ Application hanging. Our
client application will go into the tank and checking Component Services
will show that the max call time is very high and just keeps climbing.
Recycling the COM+ Application will typically fix the problem as it clears
out all memory, connections etc.

I'm struggling to figure out if our components have a bug in them or if
there is something flakey about COM+ Applications and perhaps DB Connection
Pooling.

What would cause one instance to hang and then block all other calls to a
COM+ Application?

Are there any good tools that can hack inside the low-level calls and show
what's causing the bottleneck?

Any help would be greatly appreciated. This is driving me nuts as it's
like finding a needle in a haystack.

Thanks

Eric
Brian Muth
2005-10-21 19:37:30 UTC
Permalink
Post by Eric Margheim
I have two ATL Dlls that have multiple interfaces and run under two COM+
Applications. Each DLL accesses SQL Server db via ADO calls and one of
the components also interacts with a 3rd party report server system.
We run into intermittent issues with the COM+ Application hanging. Our
client application will go into the tank and checking Component Services
will show that the max call time is very high and just keeps climbing.
Recycling the COM+ Application will typically fix the problem as it clears
out all memory, connections etc.
I'm struggling to figure out if our components have a bug in them or if
there is something flakey about COM+ Applications and perhaps DB Connection
Pooling.
This is very typical of a buggy COM+ DLL which fails to return for whatever
reason.
Post by Eric Margheim
What would cause one instance to hang and then block all other calls to a
COM+ Application?
There are a gazillion reasons. There is simply insufficient information
provided to hazard a guess what is responsible in your case.
Post by Eric Margheim
Are there any good tools that can hack inside the low-level calls and show
what's causing the bottleneck?
Having gone down this road several times myself, I've simply used standard
debugging strategies. In my design, I have built-in tracing designed as part
of the COM+ DLL's that can be turned on or off by a simply registry setting.
This produces log messages stored in a log file for analysis in case these
situations arises. This can provide a lot of insight as to what is
happening.

I really can't be more specific given the limited amount of information you
have posted, but hopefully this will keep you from going off on a tangent
and blaming the COM+ infrastructure, when it fact it is really solid (at
least in my experience).

Brian
Eric Margheim
2005-10-21 20:16:08 UTC
Permalink
Post by Brian Muth
Post by Eric Margheim
I have two ATL Dlls that have multiple interfaces and run under two COM+
Applications. Each DLL accesses SQL Server db via ADO calls and one of
the components also interacts with a 3rd party report server system.
We run into intermittent issues with the COM+ Application hanging. Our
client application will go into the tank and checking Component Services
will show that the max call time is very high and just keeps climbing.
Recycling the COM+ Application will typically fix the problem as it clears
out all memory, connections etc.
I'm struggling to figure out if our components have a bug in them or if
there is something flakey about COM+ Applications and perhaps DB Connection
Pooling.
This is very typical of a buggy COM+ DLL which fails to return for whatever
reason.
Post by Eric Margheim
What would cause one instance to hang and then block all other calls to a
COM+ Application?
There are a gazillion reasons. There is simply insufficient information
provided to hazard a guess what is responsible in your case.
Post by Eric Margheim
Are there any good tools that can hack inside the low-level calls and show
what's causing the bottleneck?
Having gone down this road several times myself, I've simply used standard
debugging strategies. In my design, I have built-in tracing designed as part
of the COM+ DLL's that can be turned on or off by a simply registry setting.
This produces log messages stored in a log file for analysis in case these
situations arises. This can provide a lot of insight as to what is
happening.
I really can't be more specific given the limited amount of information you
have posted, but hopefully this will keep you from going off on a tangent
and blaming the COM+ infrastructure, when it fact it is really solid (at
least in my experience).
Brian
Thanks for the response Brian. I am not necessarily blaming the COM+
infrastructure. Just trying to make sense of this. We have recently put
in a bunch of code to try to handle all error conditions and exceptions and
are logging them. So far nothing shows up in our logs except a sesation of
logging anything until the application is shutdown and restarted.

We have two different dlls written by different people. The commonality is
they use ADO to talk to SQL Server 2000. I can't reproduce the error in
our test environment and it doesn't happen on all of our systems and is
sporadic but very annoying.

I think what I need are tools so that when it happens I can figure out where
it's occuring and why. I also need to understand the subsystem better.

What would cause the whole application to hang if only one instance of the
component had issues? Could there be a problem with SQL Connections?
The database typically looks fine when this occurs so I don't think it's a
db call that is hanging in SQL.
Brian Muth
2005-10-21 23:02:56 UTC
Permalink
Post by Eric Margheim
What would cause the whole application to hang if only one instance of the
component had issues?
You mean you want an example? Ok, if the COM+ object requires a transaction,
then all it takes is for one object to fail or hang, and all subsequent
calls are blocked.


Could there be a problem with SQL Connections?

What sort of problem are you thinking of?

Brian
1&1
2005-10-23 13:59:48 UTC
Permalink
Hi!
Post by Brian Muth
Post by Eric Margheim
What would cause the whole application to hang if only one instance of
the component had issues?
You mean you want an example? Ok, if the COM+ object requires a
transaction, then all it takes is for one object to fail or hang, and all
subsequent calls are blocked.
Could there be a problem with SQL Connections?
What sort of problem are you thinking of?
Are you really sure that the COM+ application really "hangs"?
In the MMC only the longest running call to a component is displayed!

I remember a similiar problem with an Informix driver. Under heavy load the
COM+ application "hung" because an error in the ODBC connection pooling. Of
course it was still working but so slow that the result was that we had to
use the COM+ recycling feature.

Is it possible for you to switch off connection pooling?

btw. recycling a COM+ application does not free any resources, it starts a
new process for new callers, the existing calls still exist in the old
dllhost.exe process.

Heiko
Eric Margheim
2005-10-24 15:06:42 UTC
Permalink
Post by 1&1
Hi!
Post by Brian Muth
Post by Eric Margheim
What would cause the whole application to hang if only one instance of
the component had issues?
You mean you want an example? Ok, if the COM+ object requires a
transaction, then all it takes is for one object to fail or hang, and all
subsequent calls are blocked.
Could there be a problem with SQL Connections?
What sort of problem are you thinking of?
Are you really sure that the COM+ application really "hangs"?
In the MMC only the longest running call to a component is displayed!
I'm not sure of anything. :-) Is say "hangs" because once this happens
no calls even from new/different clients respond. I think they may open up
new instances but they don't respond. So it's certainly possible the one
call that has run the longest is blocking the others and the application
isn't really hung but I need to determine what the cause of the stoppage is.
Post by 1&1
I remember a similiar problem with an Informix driver. Under heavy load the
COM+ application "hung" because an error in the ODBC connection pooling. Of
course it was still working but so slow that the result was that we had to
use the COM+ recycling feature.
Is it possible for you to switch off connection pooling?
Not sure. I'll have to check the COM+ settings/registry.
Post by 1&1
btw. recycling a COM+ application does not free any resources, it starts a
new process for new callers, the existing calls still exist in the old
dllhost.exe process.
But they are ended, correct? If I shutdown the COM+ Application any old
calls go bye bye as far as the client is concerned so something must be
getting freed or at least trashed.

Do all calls in a COM+ Application share the same dllhost.exe? I thought
that mutiple ones would be opened?
Heiko Schulz
2005-10-27 22:11:09 UTC
Permalink
Hi!
Post by Eric Margheim
Post by 1&1
btw. recycling a COM+ application does not free any resources, it starts a
new process for new callers, the existing calls still exist in the old
dllhost.exe process.
But they are ended, correct? If I shutdown the COM+ Application any old
calls go bye bye as far as the client is concerned so something must be
getting freed or at least trashed.
Recycling a COM+ application opens a new COM+ process for new COM+ calls.
Normally the old dllhost.exe times out and stops running because itŽs
clients die.
Shutting down a COM+ application means stopping thelhost.exe process of that
COM+ aplication
Post by Eric Margheim
Do all calls in a COM+ Application share the same dllhost.exe? I thought
that mutiple ones would be opened?
Every normal COM+ application (ignore the library applications thatŽs
something special) gets itŽs own dllhost.exe process.
All calls to a COM+ application (ignore recyling) go to the same dllhost.exe
process.
Recycling opens a new dllhost.exe process for an existing COM+ application
for new callers.

Bye
Heiko
Eric Margheim
2005-10-28 04:45:58 UTC
Permalink
Post by Heiko Schulz
Post by Eric Margheim
Do all calls in a COM+ Application share the same dllhost.exe? I thought
that mutiple ones would be opened?
Every normal COM+ application (ignore the library applications thatŽs
something special) gets itŽs own dllhost.exe process.
All calls to a COM+ application (ignore recyling) go to the same dllhost.exe
process.
Recycling opens a new dllhost.exe process for an existing COM+ application
for new callers.
Cool. So when I see 20 objects spinning for the COM+ Application they are
all running under the same dllhost.exe, but each have their own memory space
to execute in?
Heiko Schulz
2005-10-30 10:14:31 UTC
Permalink
Hi!
Post by 1&1
Post by Heiko Schulz
Post by Eric Margheim
Do all calls in a COM+ Application share the same dllhost.exe? I
thought that mutiple ones would be opened?
Every normal COM+ application (ignore the library applications thatŽs
something special) gets itŽs own dllhost.exe process.
All calls to a COM+ application (ignore recyling) go to the same
dllhost.exe process.
Post by Heiko Schulz
Recycling opens a new dllhost.exe process for an existing COM+ application
for new callers.
Cool. So when I see 20 objects spinning for the COM+ Application they are
all running under the same dllhost.exe, but each have their own memory space
to execute in?
No, it seems that I confused you with too many details. ;-)

Rule of thumb:
Every COM+ application with itŽs components gets itŽs own dllhost.exe
process.

DonŽt get confused by an additional dllhost.exe process that is running for
the buildin COM+ application "System Application", it belongs to the COM+
runtime environment.

Example:
If you start your own COM+ app "MyFirstComPlusApp" youŽll notice 2 running
COM+ apps.
One dllhost.exe process for "System Application" (which you can ignore) and
another for "MyFirstComPlusApp".
All COM+ calls for "MyFirstComPlusApp" go to itŽs dllhost.exe process.
Without calls it times out and is closed. New calls start it again (with a
new process id). But you donŽt get more than 1 process per COM+ application.
But if you start another COM+ app "MySecondComPlusApp" youŽll get an
additional process for all COM+ calls of "MySecondComPlusApp".

If youŽd like to watch only the running COM+ apps open Console Root -
Component Services - Computers - My Computer - Running Processes. There you
see a per process view of the COM+ apps.
The "COM+ Application" view displays the summary of all running dllhost.exe
which is identically if you donŽt use recycling.


Now the more complicated part:
With recycling (only available for Windows servers!) itŽs possible to start
more than 1 process per COM+ app.
All exisiting calls still live in the exisiting dllhost.exe process. But new
calls go to new dllhost.exe process. If you recycle more than one time you
can get multiple dllhost.exe processes for one COM+ app. Recycling ca be
done manually or by setting limits (time, memory ...).

Bye
Heiko
Eric Margheim
2005-10-31 19:45:13 UTC
Permalink
Post by Heiko Schulz
No, it seems that I confused you with too many details. ;-)
Every COM+ application with itŽs components gets itŽs own dllhost.exe
process.
DonŽt get confused by an additional dllhost.exe process that is running for
the buildin COM+ application "System Application", it belongs to the COM+
runtime environment.
If you start your own COM+ app "MyFirstComPlusApp" youŽll notice 2 running
COM+ apps.
One dllhost.exe process for "System Application" (which you can ignore) and
another for "MyFirstComPlusApp".
All COM+ calls for "MyFirstComPlusApp" go to itŽs dllhost.exe process.
Without calls it times out and is closed. New calls start it again (with a
new process id). But you donŽt get more than 1 process per COM+ application.
But if you start another COM+ app "MySecondComPlusApp" youŽll get an
additional process for all COM+ calls of "MySecondComPlusApp".
If youŽd like to watch only the running COM+ apps open Console Root -
Component Services - Computers - My Computer - Running Processes. There you
see a per process view of the COM+ apps.
The "COM+ Application" view displays the summary of all running dllhost.exe
which is identically if you donŽt use recycling.
With recycling (only available for Windows servers!) itŽs possible to start
more than 1 process per COM+ app.
All exisiting calls still live in the exisiting dllhost.exe process. But new
calls go to new dllhost.exe process. If you recycle more than one time you
can get multiple dllhost.exe processes for one COM+ app. Recycling ca be
done manually or by setting limits (time, memory ...).
Interesting. Thanks for the information.

So if I have a situation where it seems like access to an interface or
perhaps all interfaces of my COM+ Application are hosed, this would mean the
dllhost.exe is likely in a hung state (deadlock, threading issue, etc), or
could a single instance of the component block out additional calls but
dllhost.exe itself is running fine?

At this point I'm just trying to narrow down what things, either within my
code (which makes calls to SQL Server via ADO) or within operating system
code that could cause a COM+ Application to apparently hang requiring a
shutdown of the application to clear memory out and get things working
again.

Thanks again

Eric
Heiko Schulz
2005-11-01 15:17:59 UTC
Permalink
Hi!
Post by Eric Margheim
So if I have a situation where it seems like access to an interface or
perhaps all interfaces of my COM+ Application are hosed, this would mean the
dllhost.exe is likely in a hung state (deadlock, threading issue, etc), or
could a single instance of the component block out additional calls but
dllhost.exe itself is running fine?
ItŽs possible that a single component blocks the whole COM+ app if you use
single threaded apartments.
But you should never use this apartment model for server apps, I guess you
donŽt use it.
With neutral threaded components (which can use object pooling) blocking
because of apartment boundaries shouldnŽt be your problem.

The common reason for blocking in a COM+ app is blocking of resources,
normally database locks.
Michael D. Long described some reasons for locks in this thread!
Read Serializable isolation can easily block your app if you run long
transactions.

A problem in the ODBC connection pool can also block your app. But this is
very unlikely.

Do you use your own timeouts?
Post by Eric Margheim
At this point I'm just trying to narrow down what things, either within my
code (which makes calls to SQL Server via ADO) or within operating system
code that could cause a COM+ Application to apparently hang requiring a
shutdown of the application to clear memory out and get things working
again.
1. Take care that there is no blocking in the database.
IŽm not familiar with MS SQL Server but there should be tools or a views in
the management console to watch what the sessions of your COM+ app are doing
(running SQL, locks, used resources ...).

2. Try to recycle the hanging COM+ app.
If the problem diappears after recycling (at least for new calls) you can
set the recycling time of your COM+ app. Not a fix but at least a
workaround.

3. If youŽve direct access to the server you an try to remote debug or
install a stand-alone debugger (for example WinDbg
http://www.microsoft.com/whdc/devtools/debugging/default.mspx). Then you can
attach the debugger to the running process, save a full memory dump (for
post mortem debugging) and analyze the threads. With debugging symbols
thereŽs a good chance to find the problem. At least you should see the
pattern! Set up the _NT_SYMBOL_PATH for Microsoft debugging symbols!

Some links:
http://www.codeproject.com/debug/#General
http://www.codeproject.com/debug/windbg_part1.asp
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vsdebug/html/vctskinstallingremotedebugmonitor.asp
http://blogs.msdn.com/greggm/

4. Add a test component to your COM+ app and write a caller app.
Some ideas for seperate methods that you can use to test your COM+ app when
it hangs:
- do nothing just return a fixed value => COM+ app callable?
- simple database read => database connection ok?
- simple database write => database transactions ok?
- complex database write (with 2 components) => ODBC connection pooling ok?
...

Good luck!

Bye
Heiko
Eric Margheim
2005-11-01 15:46:05 UTC
Permalink
Post by Heiko Schulz
ItŽs possible that a single component blocks the whole COM+ app if you use
single threaded apartments.
But you should never use this apartment model for server apps, I guess you
donŽt use it.
With neutral threaded components (which can use object pooling) blocking
because of apartment boundaries shouldnŽt be your problem.
Definitely not running in single threaded.
Post by Heiko Schulz
The common reason for blocking in a COM+ app is blocking of resources,
normally database locks.
Michael D. Long described some reasons for locks in this thread!
Read Serializable isolation can easily block your app if you run long
transactions.
This was my hunch too.
Post by Heiko Schulz
A problem in the ODBC connection pool can also block your app. But this is
very unlikely.
Do you use your own timeouts?
We use ADO timeouts but it's configurable and can be infinite.
Post by Heiko Schulz
Post by Eric Margheim
At this point I'm just trying to narrow down what things, either within my
code (which makes calls to SQL Server via ADO) or within operating system
code that could cause a COM+ Application to apparently hang requiring a
shutdown of the application to clear memory out and get things working
again.
1. Take care that there is no blocking in the database.
IŽm not familiar with MS SQL Server but there should be tools or a views in
the management console to watch what the sessions of your COM+ app are doing
(running SQL, locks, used resources ...).
Typically when this happens the database folks don't see any blocking on
their end. I haven't monitored waht the COM+ app is doing at the time
resource usage etc. I'll look into that. Trying to find the right
monitoring tools is half the battle.
Post by Heiko Schulz
2. Try to recycle the hanging COM+ app.
If the problem diappears after recycling (at least for new calls) you can
set the recycling time of your COM+ app. Not a fix but at least a
workaround.
When you say recycle you are referring to the operation under process view
and not Shutdown from the COM+ Application options, correct? If there are
components in call how does the recycle timeout impact things? Are you
saying that having it occasionally recycle will help free resources?
Post by Heiko Schulz
3. If youŽve direct access to the server you an try to remote debug or
install a stand-alone debugger (for example WinDbg
http://www.microsoft.com/whdc/devtools/debugging/default.mspx). Then you can
attach the debugger to the running process, save a full memory dump (for
post mortem debugging) and analyze the threads. With debugging symbols
thereŽs a good chance to find the problem. At least you should see the
pattern! Set up the _NT_SYMBOL_PATH for Microsoft debugging symbols!
http://www.codeproject.com/debug/#General
http://www.codeproject.com/debug/windbg_part1.asp
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vsdebug/html/vctskinstallingremotedebugmonitor.asp
Post by Heiko Schulz
http://blogs.msdn.com/greggm/
Thanks. I'm a noob concerning stand-alone debugging tools so this will be
great info.
Post by Heiko Schulz
4. Add a test component to your COM+ app and write a caller app.
Some ideas for seperate methods that you can use to test your COM+ app when
- do nothing just return a fixed value => COM+ app callable?
- simple database read => database connection ok?
- simple database write => database transactions ok?
- complex database write (with 2 components) => ODBC connection pooling ok?
...
Good luck!
Bye
Heiko
Heiko Schulz
2005-11-01 22:08:22 UTC
Permalink
Hi!
Post by Eric Margheim
Post by Heiko Schulz
2. Try to recycle the hanging COM+ app.
If the problem diappears after recycling (at least for new calls) you can
set the recycling time of your COM+ app. Not a fix but at least a
workaround.
When you say recycle you are referring to the operation under process view
and not Shutdown from the COM+ Application options, correct?
Yes!
When you used the term recycling in your original post you meaned stopping
and restarting the COM+ app, correct?
Post by Eric Margheim
If there are components in call how does the recycle timeout impact
things?
Recycling doesnŽt stop a COM+ app. It means starting a new process (for new
calls).
Exisiting calls stay in their "old" dllhost.exe. If a calls hangs thereŽs no
change but new calls go to the new process.
Post by Eric Margheim
Are you saying that having it occasionally recycle will help free
resources?
No, it needs more resources because of the new process. The old process will
only stop if all calls are finished or you stop it manually.
But if you recycle and new calls run fine you know itŽs really a problem in
your COM+ app and not a database server problem.

Actually we struggle with a very complex transaction problem that is caused
by a communication error between MSDTC and an ODBC driver. A very long story
that started last year and is still "developing".
To keep our COM+ apps working we had to use a very buggy .NET driver instead
of the ODBC driver.
But this driver gets problems with the connection pooling under heavy load
so we used the COM+ recycling (the setting was every 3 minutes
automatically) to avoid a hanging COM+ app. ItŽs not a solution but a
workaround!

Bye
Heiko
Eric Margheim
2005-11-02 02:26:07 UTC
Permalink
Post by Heiko Schulz
Hi!
Post by Eric Margheim
Post by Heiko Schulz
2. Try to recycle the hanging COM+ app.
If the problem diappears after recycling (at least for new calls) you can
set the recycling time of your COM+ app. Not a fix but at least a
workaround.
When you say recycle you are referring to the operation under process view
and not Shutdown from the COM+ Application options, correct?
Yes!
When you used the term recycling in your original post you meaned stopping
and restarting the COM+ app, correct?
Post by Eric Margheim
If there are components in call how does the recycle timeout impact
things?
Recycling doesnŽt stop a COM+ app. It means starting a new process (for new
calls).
Exisiting calls stay in their "old" dllhost.exe. If a calls hangs thereŽs no
change but new calls go to the new process.
Post by Eric Margheim
Are you saying that having it occasionally recycle will help free
resources?
No, it needs more resources because of the new process. The old process will
only stop if all calls are finished or you stop it manually.
But if you recycle and new calls run fine you know itŽs really a problem in
your COM+ app and not a database server problem.
Actually we struggle with a very complex transaction problem that is caused
by a communication error between MSDTC and an ODBC driver. A very long story
that started last year and is still "developing".
To keep our COM+ apps working we had to use a very buggy .NET driver instead
of the ODBC driver.
But this driver gets problems with the connection pooling under heavy load
so we used the COM+ recycling (the setting was every 3 minutes
automatically) to avoid a hanging COM+ app. ItŽs not a solution but a
workaround!
Thanks for all the great info Heiko.

Eric Margheim
2005-10-24 15:09:54 UTC
Permalink
Post by Brian Muth
Post by Eric Margheim
What would cause the whole application to hang if only one instance of the
component had issues?
You mean you want an example? Ok, if the COM+ object requires a transaction,
then all it takes is for one object to fail or hang, and all subsequent
calls are blocked.
But that would only be from the same client, correct? If a different
client with a different handle to the interface makes a call shouldn't it
have it's own process flow?
Post by Brian Muth
Could there be a problem with SQL Connections?
What sort of problem are you thinking of?
Just wondering if there could be some sort of blocking or threading issue
that would cause all SQL connections (since there is pooling going on) to be
blocked so any calls made to the components would "hang" because they are
primarliy doing db calls via ADO.
John
2005-10-23 08:36:53 UTC
Permalink
attach a debugger to the hang process and check all thread call
stack should be a fast way to check which thread makes a blocking call.
if this issue has a considerable business impact, i suggest you
submit a service request to Microsoft PSS to ask them for assistance.
"Eric Margheim" <NOSPAM******@prism-grp.com***NOSPAM> ÐŽÈëÏûÏ¢ news:u%***@TK2MSFTNGP15.phx.gbl...
I have two ATL Dlls that have multiple interfaces and run under two COM+
Applications. Each DLL accesses SQL Server db via ADO calls and one of
the components also interacts with a 3rd party report server system.

We run into intermittent issues with the COM+ Application hanging. Our
client application will go into the tank and checking Component Services
will show that the max call time is very high and just keeps climbing.
Recycling the COM+ Application will typically fix the problem as it clears
out all memory, connections etc.

I'm struggling to figure out if our components have a bug in them or if
there is something flakey about COM+ Applications and perhaps DB Connection
Pooling.

What would cause one instance to hang and then block all other calls to a
COM+ Application?

Are there any good tools that can hack inside the low-level calls and show
what's causing the bottleneck?

Any help would be greatly appreciated. This is driving me nuts as it's
like finding a needle in a haystack.

Thanks

Eric
Eric Margheim
2005-10-24 15:08:16 UTC
Permalink
John, thanks for the response. I probably will be calling MS. I'm not
sure how to attach a debugger in this production environment. Are you
referring to a stand-alone debugging tool that'll work outside of the
development environment.


"John" <***@hotmail.com> wrote in message news:OdXBhw#***@TK2MSFTNGP10.phx.gbl...
attach a debugger to the hang process and check all thread call
stack should be a fast way to check which thread makes a blocking call.
if this issue has a considerable business impact, i suggest you
submit a service request to Microsoft PSS to ask them for assistance.
"Eric Margheim" <NOSPAM******@prism-grp.com***NOSPAM> �?����Ϣ news:u%***@TK2MSFTNGP15.phx.gbl...
I have two ATL Dlls that have multiple interfaces and run under two COM+
Applications. Each DLL accesses SQL Server db via ADO calls and one of
the components also interacts with a 3rd party report server system.

We run into intermittent issues with the COM+ Application hanging. Our
client application will go into the tank and checking Component Services
will show that the max call time is very high and just keeps climbing.
Recycling the COM+ Application will typically fix the problem as it clears
out all memory, connections etc.

I'm struggling to figure out if our components have a bug in them or if
there is something flakey about COM+ Applications and perhaps DB
Connection
Pooling.

What would cause one instance to hang and then block all other calls to a
COM+ Application?

Are there any good tools that can hack inside the low-level calls and show
what's causing the bottleneck?

Any help would be greatly appreciated. This is driving me nuts as it's
like finding a needle in a haystack.

Thanks

Eric
Michael D. Long
2005-10-29 21:23:42 UTC
Permalink
Response inline...
--
Michael D. Long
Post by Eric Margheim
What would cause one instance to hang and then block all other calls to a
COM+ Application?
When operating against the DBMS at Read Serializable isolation, a long
running transaction that acquires an exclusive lock on a single data row
will block all other connections accessing the same data row. For that
reason, when developing COM+ applications the developer needs to be
intimately familiar with database locking behavior and data usage patterns.

I saw one guy try to avoid a scalability issue by creating a connection
outside of the scope of the distributed transaction in an effort to improve
performance on a system designed by another team that was bottlenecked at 8
concurrent users. This approach managed to "intermittently" deadlock the
entire application, because the non-transactional connection was contending
with the connection enlisted in the distributed transaction for the same
data row - and his internal recovery logic kept retrying until the object
was terminated.
Post by Eric Margheim
Are there any good tools that can hack inside the low-level calls and show
what's causing the bottleneck?
If this is a data related bottleneck you can run a trace of the activity. A
thorough design review wouldn't hurt - this should include validation of the
data access patterns as well as line by line source code walk through.
Eric Margheim
2005-11-01 20:14:24 UTC
Permalink
Post by Michael D. Long
When operating against the DBMS at Read Serializable isolation, a long
running transaction that acquires an exclusive lock on a single data row
will block all other connections accessing the same data row. For that
reason, when developing COM+ applications the developer needs to be
intimately familiar with database locking behavior and data usage patterns.
I saw one guy try to avoid a scalability issue by creating a connection
outside of the scope of the distributed transaction in an effort to improve
performance on a system designed by another team that was bottlenecked at 8
concurrent users. This approach managed to "intermittently" deadlock the
entire application, because the non-transactional connection was contending
with the connection enlisted in the distributed transaction for the same
data row - and his internal recovery logic kept retrying until the object
was terminated.
Michael would this be detectable from looking at the SQL Server database or
could it appear quiet on the db end with the problem lying wihtin the ADO
calls in the component? I ask because I know that SQL server isn't locking
according to our database team when this happens.
Post by Michael D. Long
Post by Eric Margheim
Are there any good tools that can hack inside the low-level calls and show
what's causing the bottleneck?
If this is a data related bottleneck you can run a trace of the activity. A
thorough design review wouldn't hurt - this should include validation of the
data access patterns as well as line by line source code walk through.
Thanks for the suggestions.
Loading...