JIT compilation in MATLAB

Michael Gelbart · May 13, 2013

A few years ago MATLAB introduced a Just-In-Time (JIT) accelerator under the hood. Because the JIT acceleration runs behind the scenes, it is easy to miss (in fact, MathWorks seems to intentionally hide it so that users do not change their coding style, probably because the JIT accelerator is changed regularly). I just wanted to briefly mention what a JIT accelerator is and what it does in MATLAB. JIT compilers compile chunks of code at runtime, so instead of interpreting the MATLAB code line-by-line (the default for an interpreted language), it compiles certain chunks of code and then runs the compiled chunks as a block. The JIT acceleration in Matlab works quite well. I wrote the following code to demonstrate MATLAB's JIT in action:

N = 1e7;

w = zeros(1, N);
for i = 1:N
    w(i) = i*5;
fprintf('For loop: %.4fs\n', toc);

y = (1:N)*5;
fprintf('Vectorized: %.4fs\n', toc);

The code takes the integers from 1 to 10 million and multiplies each one by 5. When I run this on my machine (Mac Mini), the output is:

For loop: 0.0590s
Vectorized: 0.0720s

This means that the "for" loop is faster than the vectorized code, contrary to the conventional wisdom that loops are much slower than vectorized code in MATLAB. Now I want to try the same thing without any JIT acceleration. To do this, I type feature('accel','off') into the MATLAB Command Window and run the code again. Doing so gives the following results:

For loop: 19.4777s
Vectorized: 0.0487s

I can't vouch for why the vectorized code actually went faster in this run, although the effect seems to be reproducible (perhaps there is a fraction of a second overhead trying to accelerate, which is wasted when the code can't be accelerated). However, the important point is that the loop implementation is slowed down by more than 300x when acceleration is turned off. This is JIT acceleration at work (or not at work, in this case).
[Note: if you are following along, don't forget to type feature('accel','on') to turn the accelerator back on when you're done!]

Having said all this, I still suggest writing vectorized MATLAB code whenever possible. The JIT accelerator is fragile and some fairly simple operations can cause the accelerator to skip over a loop. Still, I wanted to bring this to your attention, at least to help you avoid embarrassing situations like one I had a few years ago as a TA. I was giving a MATLAB tutorial to freshmen at Princeton and I was preaching the virtues of writing vectorized code. I decide to do a demonstration much like the one above (but with computing Fibonacci numbers, using a loop vs. vectorized using the Fibonacci formula). To my embarrassment the demo failed completely: the loop performed better than the vectorized code (and avoided overflow/underflow errors to boot!). That is how I first discovered MATLAB's mysterious JIT accelerator.

Twitter, Facebook