Processing Interruptions blocking Serial Comm

SamCarlson

Hello,

I have programmed a laser locking system on the quarto for atomic physics research. Everything compiles clean, but after testing, I've learned that I've hit some kind of computational resources limit. As it is, the program is not connecting to Tera Term correctly. However, commenting out various sections of the code (different chunks every time) allows the serial port connection. I'm getting no errors or warnings when I upload the code to the quarto, and the compiler says that global variables are only using 13% of the memory. I'm very stuck and not sure how to move forward. Would you please be able to take a peek at the code or point me in the right direction?

Ben

When processing in interrupts (typically ADC data) you can get into trouble if the processing takes too long. Basically if you get ADC data every, say, 5us, and it takes 6us to process that data, then the the Quarto can't keep. Once it finishes the interrupt to handle the ADC, it will immediately run that same routine to process the next data and it will ignore all other processing. The Quarto should detect this and reboot itself into the bootloader. The front panel LED will flash blue/red to indicate that there was a lock-up and that it rebooted. Because the Quarto is back in the bootloader, you will be able to upload new code to it.

Here's simplified code the shows the issue:

const uint SampleRate = 2;

void setup(void) {
  configureADC(1, SampleRate, 0, BIPOLAR_1250mV, getADC1);
  configureADC(2, SampleRate, 1, BIPOLAR_10V, getADC2);
}

void loop(void) {
  static unsigned long lastrun;
  if (millis() > lastrun) { //Run once every 500ms
    lastrun = millis() + 500;
    toggleLEDGreen();      
  }
}

void getADC1() {
  double adc1 = readADC1_from_ISR();
  //some code that may be slow
  ...
}

void getADC2() {
  double adc2 = readADC2_from_ISR();
  //some other code that may be slow
  ...
}

If this code is causing the Quarto to reboot, two suggestions for troubleshooting. The first is to slow it down and confirm that that fixes the problem. In this case, we change SampleRate from 2 to 10 so we have five times the time for processing. If that doesn't fix the rebooting, you can increase it further, or look into other issues. The next change is to start timing the critical functions. The easiest way to do this is with the triggers. We will have Trigger 1 go high when running the getADC1 and Trigger 2 go high when running getADC2:

const uint SampleRate = 10;

void setup(void) {
  configureADC(1, SampleRate, 0, BIPOLAR_1250mV, getADC1);
  configureADC(2, SampleRate, 1, BIPOLAR_10V, getADC2);
  triggerMode(1,OUTPUT);
  triggerMode(2,OUTPUT);
}

void loop(void) {
  static unsigned long lastrun;
  if (millis() > lastrun) { //Run once every 500ms
    lastrun = millis() + 500;
    toggleLEDGreen();      
  }
}

void getADC1() {
  triggerWrite(1,HIGH);
  double adc1 = readADC1_from_ISR();
  //some code that may be slow
  ...
  triggerWrite(1,LOW);
}

void getADC2() {
  triggerWrite(2,HIGH);
  double adc2 = readADC2_from_ISR();
  //some other code that may be slow
  ...
  triggerWrite(2,LOW);
}

This this code, if we look at the triggers on an O-scope we might see something like:

Where the first interrupt (getADC1) consistently takes _500nS to complete and the second (getADC2) always starts 1us after the first, but takes between 1 and 2us to complete. This variation could be in the code (if statements that sometimes execute) or some functions, like sine and cosine, take a variable amount of time to complete depending on the input values.

Ideally the executing time would always be shorter than the frequency to want to run the function. But if your code usually takes 2.5us to complete, but occasionally takes 3.2us, it could run every 3us. Whenever the the 3.2us completion time happens, the processing of the next ADC value gets delayed but if that next execution time is 2.5us, it can catch up to be ready on time for the next one. But the average processing time needs to be less than the frequency of execution.

Ben

A couple thoughts on improving the timing:

The first step is to understand what function(s) are slow. You can use the same trick with the triggers to time different parts of the function. If you have a slow calculation, check to see if you can pre-calculate in setup() and store the results and then the interrupt function can just read the result from that stored result instead of recalculate it. Also, depending on the application, may it make sense to skip some processing when the processor is getting behind.

Ben

One common source of 'slow' functions are trig functions. Sine, cosine and the like are very accurate calculations that take a variable amount of time, which is why the above plot shows a lot of timing variation. A quick sin approximation function that runs much faster is:

double fast_sin(double x) {
    const double overTwopi = 1/(2*PI);
    x *= overTwopi; //divide by 2*PI, but structure as multiplication
    x -= (int) x;
    if (x <= 0.5) {
        double t = 2 * x * (2 * x - 1);
        return (PI * t) / ((PI - 4) * t - 1);
    }
    else {
        double t = 2 * (1 - x) * (1 - 2 * x);
        return -(PI * t) / ((PI - 4) * t - 1);
    }
}

Using that in the code example that generated the previous plot gives faster and more deterministic timing:

Now that looks like it takes about 2.3us to execute both interrupts, but once new data comes in faster than 2.3us, it will delay the code execution and that dead-time between the two pulses will decrease, so the actual processing time is less than 2.3us. To see how fast the combo actually executes, we need to switch the order those two ADC's run (change the phase on the configureADC function), then the timing plot looks like:

and it takes 1.8us to process, so the loop can now support a SampleRate of 2.

In a future release, we will add fast, but approximating Trig functions to make it easier to do these kinds of tricks.