Well, we figured out the problem with the 1296 rig, but I'm unsure how to fix the problem. Apparently there has been no oscillation problem all along, the local oscillator chain is producing harmonics that are causing some strange mixing products. We had a strong output on around 1250 mhz that was always at the output anytime the transverter was keyed. This signal goes away when the oscillator is detuned so that it stops oscillating, but is always there anytime the oscillator is going. The signal levels are equal to the desired 1296 signal when driven by the IF rig. So when the rig is running, I had an output on 1296 and a carrier at 1250 (or thereabouts). We traced the problem to a dirty LO chain.
We were able to clean it up substantially by inserting a 3 pole BPF between the 10 mW transverter output and the next amplifier stage. The correct way to fix this in my opinion would be to incorporate another of these filters in the output of the LO chain. We didnt get that far with it today though.
When we hooked up the 2W PA, there was no output from the amp, and we didnt have time to troubleshoot it. It was working when we began testing, so I am guessing it is something simple, such as a short in the RF path somewhere. Since the TX is now clean, I can now troubleshoot the amp and try to maximise the output power.
Another possible problem with the dirty LO is that of receive performance. With a dirty LO, the receiver will be receiving on 2 different frequencies, cutting the gain down by 3dB (or more) on the desired 1296 MHz, and also increasing the noise by 3 dB (or more), I never really noticed a receive problem with the exception of the RF stage oscillating, which was a confirmed problem. So, with that discovery, work can begin cleaning up the LO, and eventually I should get a 1296 rig on the air , hopefully before summer.