1. Hard disable – when the plugin causes the host (Excel) to crash

In this case, the COM addin will be in the disabled list and the user will get a message next time it starts Excel asking whether the user still wants that addin or not.
The way Excel does this is by putting the plugin on a black list before calling the OnConnection method (IDTExtensibility2). If the method crashes the host, next time Excel starts up, it finds the offending plugin in the black list.
If the method returns fine, Excel removes it from the black list.
It’s difficult to crash the host from .net, but not impossible. Most of the disables in .net are however…

2. Soft disable – when the plugin returns a failure HRESULT from OnConnection

In .net that means that an exception escapes from OnConnection. The .net framework converts this into an HRESULT when the method returns.
In order to fix that, you need to look into the following areas:
- catch every exception in OnConnection (duh!)
- look at any members part of your class implementing IDTExtensibility2 (any constructor there which throws will generate this condition)
- also look at any static members that can potentially throw
- look at any problems loading dependent assemblies needed by your assembly (this can also cause this)
- if you haven’t written a wrapper for your plugin, and your plugin runs as the mscoree.dll plugin, then check if other similar plugin haven’t been disabled (Excel disables mscoree which disables all plugins running under that – check the MS website on how to write a wrapper and avoid this problem)

So the first really easy way to catch this is check if you hold the lock, if not, easy, just add it.

The other situation is to lock one object and operate on another:

lock(Object1)
{
    if(condition)
    {
        Monitor.Wait(Object2)
    }
}

These scenarios are easy to catch and fix because they throw all the time. But this exception can occur in race conditions where it’s more difficult to catch. Let’s look at the following code:

Thread 1 (publisher):

lock(_queue)
{
     _queue.Add(new_element);
     Monitor.Pulse(_queue);
}

Thread 2 (consumer):

List queue;
lock(_queue)
{
     queue = _queue;
     _queue = new List();
}
ProcessQueue(queue);

The idea here is to try to optimize the way the list of type T is emptied and then processed. In particular, the optimization is that it doesn’t create a copy. The copy can be re-written, without this optimization as:

     queue = new List(_queue);
     _queue.Clear();

This has the problem that it allocates memory and copies data, all with the lock held.

But unfortunately, while the optimization works, it’s not entirely correct, as it created a hidden race condition. To illustrate the race condition, I’ll present it here, step by step, with the code running on both threads interlaced below:

1: Thread2:  lock(_queue) <- acquires lock on _queue
2: Thread1:  lock(_queue) <- lock is held by thread2, so thread1 gets blocked here
3: Thread2:  queue = _queue;
4: Thread2:  _queue = new List();
5: Thread2:  release lock on _queue
6: Thread1:  _queue.Add(new_element);
7: Thread1:  Monitor.Pulse(_queue);

At this step 7, Monitor.Pulse throws a System.Threading.SynchronizationLockException.
The reason is that thread1 acquired a lock on _queue but then thread2 changed the “meaning” of the reference _queue, so when thread1 pulses the _queue, it pulses a different object than the one it acquired the lock on. Thus the exception.

Once the problem is understood, the solution is simple: synchronize on an immutable object. You can create a dummy object and sync on that, rather than the mutable _queue.
Or you can implement the more inefficient version, which doesn’t actually change the object _queue refers to.

PS: To reproduce this relatively consistently, insert a Thread.Sleep(5000) in thread2, right after it acquired the lock.

This can be easily replicated by starting Excel in automation mode:

Excel /automation spreadsheet_name.xls

Using the UDF in the automation addin has the same problem, it shows #NAME?

After a bit of googling, it turns out that in automation mode, you have to re-enable the plugin as it’s not loaded by default. So the solution is to un-tick the addin in the Tools/Addins dialog and tick it back on.

Or programatically you need to do this:

set oAddin = Application.Addins(“YourAddin.ProgId”)
oAddin.Installed = False
oAddin.Installed = True

If it’s not in the list, you will have to add it first with:

Application.Addins.Add(“YourAddin.ProgId”)

Once you do that, the UDF starts working fine.

If your UDF is in an XLL rather than an Automation Addin, then you can also use the RegisterXLL API.

In the mean time, I was doing quite a bit of Excel programming and that’s the experience I want to talk about. I didn’t know much about Excel before starting. I have done VB, VBScript before so that came as no surprise. I also knew Excel is used quite a lot in the financial community, so I was keen to get some exposure to it.
After almost a year of development, ranging from VBA, plugin development (COM addin + Automation addin – DDE) and RTD, I can say, I have mixed feelings.

The overall experience I’m taking from this is that Excel is not a mature environment for development and the experience can be quite frustrating. It is almost decent at VBA, but when you start with addins, RTD, it all leaves a slightly bitter taste in your mouth. And here’s why, all the reasons listed one after the other:

1. Excel is single threaded. This has several implications:
a. You cannot call into Excel when it’s busy, RPC calls fail, for this you have to retry, or sometimes you get the dreaded dialog telling you that the STA is busy
b. It’s slower than it should be. Cannot take advantage of multi-processors
c. Can get easily into a “frozen” state, when long operations have to be done on the UI thread
d. Operations need to be marshalled into the STA, which is extremely slow and I do mean extremely

In Excel 2007 they have introduced multi-threading. However, they have also changed the interface, introducing the ribbon interface and alienating the entire financial community. No one is moving to Excel 2007 for that reason.

2. Error handling is extremely poor

When something happens, you don’t get a clear error, but instead a general error of what happens. Many APIs return the same errors and the errors are not documented.
This is truly a killer, and it’s true with the whole Excel object model. A very very big issue.

3. The addin frameworks are all a bit of mess

There are 3 main types of addins:
a. COM addin, you can implement UI elements, but no UDFs
b. Automation addins, you can implement UDFs, but no UI elements
c. RTD servers, you can implement a very limited interface to update data, a pull-push hybrid, but no access to the Excel object model at all

So, let’s say you want to implement an addin which has a UI menu with some functionality, provides a UDF, which operates asynchronously. Then you need a combination of potentially all three addins in one, maybe add VBA xla on top of that.

4. Some operations don’t work from addins, only from VBA, or only from some context, or only if the VBA_E_IGNORE is not set, or if other black box conditions are set

Again, lack of documentation makes it really difficult to develop. It’s a trial & error approach which works most of the time.
You always face the problem where your addin works most of the time, unless the user does something which you didn’t expect and tested.

5. Pushing data into Excel is very difficult

Either Excel is busy, or sets itself into VBA_E_IGNORE when a dialog is present, or operations fail because you’re in edit mode.
This problem was addressed by the introduction of the RTD framework

6. Excel is still buggy and it can crash

If you do certain operations, you might hit a bug in Excel and crash it. It crashes because bugs in Excel, but your addin/VBA are triggering, so you look like a culprit.

7. Excel doesn’t perform very well when you have big workbooks with a lot of data

8. Very difficult to debug problems in Excel, or weirdness like above, because Excel is a black box

Still, Excel is loved in the financial community, because it makes non-programmers more productive by its operations. It’s an easy way to present, format data which can be refreshed and kept up to date.
Yet, for developers, it can be a nightmare.

Two MIT students have managed to take pictures or the Earth curvature by launching a photo camera + mobile phone attached to a balloon. I was really impressed. If I was a kid, this would be very motivating stuff to get me into science. I would probably try to replicate it too…

Being a software engineer, I’m thinking … what if you get a “smartphone” and you write some software for it … what software would you write? Well, let’s see:

1. Software to control the camera integrated in the smartphone, so you don’t need a camera and a phone
2. Software to connect via gprs and submit live images (or hey, a live streaming) … question is, would you have any signal at 28km above the earth?
3. Software to monitor the GPS trajectory and altitude, so that you can plot it on a map and you can draw all sorts of graphs
4.  A IM chat client to tell your friends you’ve done it

What can you do with it? You can write an Expert Advisor, this is a algo trading application.
It has a 3 methods: init, deinit and start. The first 2 act as constructor/destructor, whereas the second is called on every tick, which you can process and then decide whether to open a trade, close a trade or do nothing.

Most companies look at it as a purely tactical move, although they call it strategic. The attraction is cost. However, because the move is tactical, the integration is also at a very tactical level. Usually the team assigns unattractive tasks like defect fixing or other uninteresting work to the offshore teams. Which ultimately leads to other problems like lack of knowledge, specialization and ultimately productivity (which is labelled as failure, and rightly so). Also, because the work is so un-attractive, it only appeals to mediocre employees, the best ones leave for greener pastures. In the end, it becomes a problem of loyalty.

An important problem is communication and the fact that there’s a bigger disconnect betwen the onshore and offshore (or nearshore) teams. Specs have to be more detailed and knowledge needs to be transferred more formally (which in IT doesn’t work that well – a lot of knowledge is tacit and not necessarily easy to capture in documents and diagrams).
This in the end results in more tech lead and PM time, which is quite expensive. Tech leadership and PM become full time jobs dedicated to the offshore team. If you add the cost of lower productivity to do with poorer communication and disconnection from the business/front office and you add also the travel costs, I’m not sure it’s such a good deal anymore.

Nevertheless, waves and waves of management professionals and consultants will fall for it, because on paper it represents such a good deal. You can boast about cutting cost which is music to the ear for the upper management, especially in recessions. And then they can cash in their bonuses and claim success in the short term, whereas the company (and shareholders ultimately) are left to pick up the pieces.

I’m not against outsourcing though, I’m just against it when it’s done for the wrong reasons. I also think that doing it for the right reasons would also mean doing less of it overall. Just because it’s not a universal panacea.

The equation is:

m = (360 * Yd) / (360 – t * Yd)

where m is the Money Market equivalent yield, Yd is the discount yield, t is the number of days between the settlement and maturity.

This equation is required when you want to compare the return of a tresury bill (quoted in discount yield) and an equivalent CD (certificate of deposit). The day count convension for both is Act/360.

The discount yield for a t-bill is calculated with the following:

Yd = D * 360 / (t * F)

where D is the cash discount (actual amount of money that you get after t days per face value unit) and F is the face value.

The price of the t-bill is then P = F – D, assuming usually a face value of 100$.

The T-bill cash flow after t days is D (the discount). For a CD, the cash flow after t days is m * P * t / 360 where m is the money market yield and P is the price (or money invested).

To calculate equivalent yields, the cashflows must match. So, we have:

D = m * P * t / 360

Resolving for m, results into:

m = D * 360 / (t*P)

We extract D from the definition of the discount yield:

D = Yd * F * t / 360 

which gives:

m = Yd * F / P

P = F – D = F – Yd * F * t / 360 = F ( 1 – Yd * t / 360) = F * ( 360 – Yd * t) / 360

We replace the price in the equation for m:

m = Yd * 360 / (360 – Yd * t)

which is the equation in Fabozzi.

This conversion is used to compare T-bill investments with money market investments because the discount yield cannot be directly compared with the money market yield.