Archive for October, 2010

System.Threading.SynchronizationLockException

October 20, 2010

I got this exception a while ago when implementing a simple publisher/consumer scheme with 2 threads in C#. What this exception says is that you’re trying to use some operations without a lock held. In my case it was Monitor.Pulse(object).
These sort of operations conceptually need to be done with the lock on object held, for the same reason old Unix condition variable had to do certain operations with the lock held: to avoid race conditions where one thread can miss the signal and get blocked forever on a synchronization primitive.

So the first really easy way to catch this is check if you hold the lock, if not, easy, just add it.

The other situation is to lock one object and operate on another:

lock(Object1)
{
    if(condition)
    {
        Monitor.Wait(Object2)
    }
}

These scenarios are easy to catch and fix because they throw all the time. But this exception can occur in race conditions where it’s more difficult to catch. Let’s look at the following code:

Thread 1 (publisher):

lock(_queue)
{
     _queue.Add(new_element);
     Monitor.Pulse(_queue);
}

Thread 2 (consumer):

List queue;
lock(_queue)
{
     queue = _queue;
     _queue = new List();
}
ProcessQueue(queue);

The idea here is to try to optimize the way the list of type T is emptied and then processed. In particular, the optimization is that it doesn’t create a copy. The copy can be re-written, without this optimization as:

     queue = new List(_queue);
     _queue.Clear();

This has the problem that it allocates memory and copies data, all with the lock held.

But unfortunately, while the optimization works, it’s not entirely correct, as it created a hidden race condition. To illustrate the race condition, I’ll present it here, step by step, with the code running on both threads interlaced below:

1: Thread2:  lock(_queue) <- acquires lock on _queue
2: Thread1:  lock(_queue) <- lock is held by thread2, so thread1 gets blocked here
3: Thread2:  queue = _queue;
4: Thread2:  _queue = new List();
5: Thread2:  release lock on _queue
6: Thread1:  _queue.Add(new_element);
7: Thread1:  Monitor.Pulse(_queue);

At this step 7, Monitor.Pulse throws a System.Threading.SynchronizationLockException.
The reason is that thread1 acquired a lock on _queue but then thread2 changed the “meaning” of the reference _queue, so when thread1 pulses the _queue, it pulses a different object than the one it acquired the lock on. Thus the exception.

Once the problem is understood, the solution is simple: synchronize on an immutable object. You can create a dummy object and sync on that, rather than the mutable _queue.
Or you can implement the more inefficient version, which doesn’t actually change the object _queue refers to.

PS: To reproduce this relatively consistently, insert a Thread.Sleep(5000) in thread2, right after it acquired the lock.

Advertisements

Excel: UDF not available in automation mode

October 19, 2010

One problem I was having lately is that some UDF in an Automation addin was not available when Excel was started in automation mode, via a C# application. Checking the Automation addin in Tools/Addins showed it was enabled, ticked and should have been working.
But when trying to call the UDF, I was getting #NAME?

This can be easily replicated by starting Excel in automation mode:

Excel /automation spreadsheet_name.xls

Using the UDF in the automation addin has the same problem, it shows #NAME?

After a bit of googling, it turns out that in automation mode, you have to re-enable the plugin as it’s not loaded by default. So the solution is to un-tick the addin in the Tools/Addins dialog and tick it back on.

Or programatically you need to do this:

set oAddin = Application.Addins(“YourAddin.ProgId”)
oAddin.Installed = False
oAddin.Installed = True

If it’s not in the list, you will have to add it first with:

Application.Addins.Add(“YourAddin.ProgId”)

Once you do that, the UDF starts working fine.

If your UDF is in an XLL rather than an Automation Addin, then you can also use the RegisterXLL API.

Excel

October 9, 2010

I know it’s been a long time since my last post and I apologize.

In the mean time, I was doing quite a bit of Excel programming and that’s the experience I want to talk about. I didn’t know much about Excel before starting. I have done VB, VBScript before so that came as no surprise. I also knew Excel is used quite a lot in the financial community, so I was keen to get some exposure to it.
After almost a year of development, ranging from VBA, plugin development (COM addin + Automation addin – DDE) and RTD, I can say, I have mixed feelings.

The overall experience I’m taking from this is that Excel is not a mature environment for development and the experience can be quite frustrating. It is almost decent at VBA, but when you start with addins, RTD, it all leaves a slightly bitter taste in your mouth. And here’s why, all the reasons listed one after the other:

1. Excel is single threaded. This has several implications:
a. You cannot call into Excel when it’s busy, RPC calls fail, for this you have to retry, or sometimes you get the dreaded dialog telling you that the STA is busy
b. It’s slower than it should be. Cannot take advantage of multi-processors
c. Can get easily into a “frozen” state, when long operations have to be done on the UI thread
d. Operations need to be marshalled into the STA, which is extremely slow and I do mean extremely

In Excel 2007 they have introduced multi-threading. However, they have also changed the interface, introducing the ribbon interface and alienating the entire financial community. No one is moving to Excel 2007 for that reason.

2. Error handling is extremely poor

When something happens, you don’t get a clear error, but instead a general error of what happens. Many APIs return the same errors and the errors are not documented.
This is truly a killer, and it’s true with the whole Excel object model. A very very big issue.

3. The addin frameworks are all a bit of mess

There are 3 main types of addins:
a. COM addin, you can implement UI elements, but no UDFs
b. Automation addins, you can implement UDFs, but no UI elements
c. RTD servers, you can implement a very limited interface to update data, a pull-push hybrid, but no access to the Excel object model at all

So, let’s say you want to implement an addin which has a UI menu with some functionality, provides a UDF, which operates asynchronously. Then you need a combination of potentially all three addins in one, maybe add VBA xla on top of that.

4. Some operations don’t work from addins, only from VBA, or only from some context, or only if the VBA_E_IGNORE is not set, or if other black box conditions are set

Again, lack of documentation makes it really difficult to develop. It’s a trial & error approach which works most of the time.
You always face the problem where your addin works most of the time, unless the user does something which you didn’t expect and tested.

5. Pushing data into Excel is very difficult

Either Excel is busy, or sets itself into VBA_E_IGNORE when a dialog is present, or operations fail because you’re in edit mode.
This problem was addressed by the introduction of the RTD framework

6. Excel is still buggy and it can crash

If you do certain operations, you might hit a bug in Excel and crash it. It crashes because bugs in Excel, but your addin/VBA are triggering, so you look like a culprit.

7. Excel doesn’t perform very well when you have big workbooks with a lot of data

8. Very difficult to debug problems in Excel, or weirdness like above, because Excel is a black box

Still, Excel is loved in the financial community, because it makes non-programmers more productive by its operations. It’s an easy way to present, format data which can be refreshed and kept up to date.
Yet, for developers, it can be a nightmare.