The Advantages of Matrix Math Computer Languages

Linux Offers a Gold Mine of Matrix Languages

Why? For one -- Linux, and all it contains, is free (listen up college students). For another, Linux and most of the support for it is open source. But in this case, mostly you should consider Linux because when it comes to mathematics languages -- Linux is loaded.

What Do Matrix Languages Offer?

That brings us to scripting matrix oriented languages. They can grow on you. For years I avoided them, largely because I knew that they were really memory hogs, and only a few years ago PCs didn't have all that much memory. Not enough, for example, to hold an entire time series of some space craft data, plus all the intermediate matrices that might accumulate when you tried processing said space craft data. But as PCs grew, I moved into the matrix realm to try it out. For most of my math needs, I've never moved back.

I started with octave, then went to scilab. Then I discovered R, followed by yorick, and finally pdl. Unable to decide on just one, I worked with the primitive I/O commands in each language to create a simple binary file format that could be created by each of the languages, and read by each of them. This allowed me the luxury of being able to begin processing on one language because of some specific capability it had, and move to another for some other desirable property. I still work that way. It was tough at first, but now saves me enormous time.

To make moving across languages easier, I created libraries for each that had the same names, like fileio, math, and plots. Within those libraries, I created routines to do what I wanted, and named them the same in each language also, like binall for the binary input routine, or plot3d for plotting a 3D image. In some cases, I was just basically renaming the specific language's routine that already did what I wanted, but often I was using the language's functions in combinations or with specific parameters to accomplish what I wanted. Thus armed with similar libraries, with similar or identical functions for familiar results, I am able to move from language to language with little discomfort.

Each of the languages I'm familiar with has capabilities similar to those of the popular commercial MATLAB language. Some are a little less populated with math functions than MATLAB, but some are very robust in support. All are matrix oriented in design. Each language, while bearing much in common to the others, has some emphasis or particular capabilities that make it shine for some kind of problem. I suggest that when it comes to these languages, you don't let syntax familiarity overwhelm your considerations. Sometimes a particular capability or formulation trumps syntax in the tackling of a difficult problem.

If you're not overly familiar with matrix languages, you may be wondering "What's the big deal." Why not just use BASIC, or Fortran, or C, or some other general purpose language?

One Big Advantage is the Scripting

The main attraction of these languages is the interactive and scripting capabilities they offer. They satisfy the desire to be able to quickly prototype code without the time consuming compiling and linking that goes with a more fundamental language. It's also handy to be able to interactively try out code examples to see how they work before committing them to a code file. For example, you can interactively test how a particular filter or graphics routine works.

The problem with scripting languages is that an engine that processes the script has to read, interpret and act upon each command. This takes time -- much more time than a compiled program takes.

However, if you have compiled code in your engine that works on large blocks of numbers (matrices), then one interpreted command can result in compiled code speed for performing a processing command on a large data block. So if one can learn to express the solution to his or her problem with matrix notation, the number of script entries versus the number of calculations made is trivial, giving performance near that of compiled programs.

That's why matrix oriented scripting languages are so popular. That plus the fact that most of them have easy to use input and output commands for saving entire programs or matrices, and easy to use plotting commands for graphing the contents of matrices in many ways. The end result is much less code to develop, some interactive capabilities, yet quite reasonable execution speed.

The Trick is to Vectorize Your Solution

The trick is to be able to express math solutions in matrix nomenclature. That means -- avoid do loops whenever possible. One very good guideline, documented with the yorick language distribution, is that if your solution operates on a record at a time and could be operated on forward or backward with the same result, it can likely be expressed in matrix nomenclature.

Another way to express this is that if each record of computation is only dependent on data in that record (no dependence on previous records as in integration), then it can be expressed in matrix nomenclature. That means it can be processed in a matrix scripting language with tolerable speed. Note that for prior record dependencies, most languages have a few built in compiled routines to handle common problems of that type, like cumulative summing, or integration.

As an example, assume you have a file with a number of x and y values, and you want to read them in and compute a least squares line (y =mx + b). In a conventional language this would entail some program preamble statements, file opening statements, file reading statements, loops to compute the necessary sums, and finally computation of the result.

In a matrix language, like octave, this could be done interactively as simply as:

Get the idea? The matrix operators cov (covariance), var (variance) and mean inherently contain loops to do the computations, returning the results. Once you catch on the the matrix formulation of problem solution, you'll likely find that you can save lots of time using a matrix scripting language.

Another hint is to think about matrix dimensionality. Remember the property of matrix multiplication that an l x m matrix times an m x n matrix results in an l x n matrix. The first size indicates the number of rows in a matrix, and the second the number of columns. Sometimes just studying what matrix sizes you have and what size you want to end up with helps identify the steps you need to do to accomplish what you want. You may need to construct a special multiplier matrix to create the result you want, but when you see how to do it you'll be on the way to vectorizing your solutions so you can solve problems with matrix operations and no loops.

A Speed Comparison of Some Matrix Languages

To do a quick speed test, I generated a million records of a function and its component functions in each of 7 different matrix languages. I then had each language perform a least squares solution to recover the scale factors of the components of the generated function. The time for each language to loop through the data generation and least squares solution 4 times was logged. Full vectorization was used in each language for both data generation and least squares solution, so no loops were used in the scripting code.

The model function was:

With i running from 1 to 1000000

This exercise tested both the languages' ability to compute large numbers of transcendental functions as well as the matrix operations used to perform the least squares solution. Each test was performed for the same number of data points and iterations, and on the same computer. The following code snippet shows the matrix operator method of creating the large data matrix:

The next code snippet shows the use of a loop to create the large data matrix:

In the following table, the Matrix Time column shows the program run time in each scripting language when only matrix operators were used to create the data and perform the least squares solution. The Loop Time column shows the program run time that resulted when the large data matrix was repeatedly formed using a loop procedure. The Time Ratio is the Loop Time divided by the Matrix Time, showing the ratio of times just from looping instead of using matrix operations.

The table shows that not only did the languages do an enormous amount of work in little time when optimized to use the built in matrix functions, but also that the languages weren't really separated by that much in performance.

The table also illustrates that if the scripts didn't make optimal use of matrix operations and resorted to loops, the cost for some was heavy. The Perl PDL, Yorick, and Tela languages seemed most forgiving in this regard. Euler showed a moderate slowing when loops were used, and R, Octave, and Scilab were much slower when loops were heavily used.

Certainly, except in special conditions, speed alone isn't the only consideration for choosing a language. It has a lot to do with what overall use the language will be applied, resource use, and even the programmer's background and preferences. For that reason, I prepared another chart that considers some other factors of several scripting languages.

A Qualitative Comparison of Some Matrix Languages

The graphic you see above presents a comparison of a number of math languages I have worked with in Linux. Some of these languages are also available in Windows, but my primary experience with them is in Linux.

Any such graph or table is subjective, and I admit that these are my own personal appraisals. If anything, I've probably underrated some of the languages in that I'm not an expert in all of them. But I have used all of them and exercised many of their features.

I did not concentrate on syntax and overall features in making this comparison. All of these languages are matrix oriented programs very capable of solving a wide range of mathematical problems. They overlap considerably in that regard, with some having a bit more extension into statistics (like R) or engineering (like Octave and Scilab).

Instead I concentrated on features that seemed to separate the languages. The features I chose are ones that I think can help provide a direction of choice for many prospective users. I subjectively rate the languages on their speed, I/O flexibility, support of higher level data structures (beyond matrices), and ability to work with strings.

You might think speed is an attribute that could be tested and reported quite objectively. The problem is the languages express their speed differently in different kinds of tests. Some, like PDL, are extremely fast at solving common matrix problems. In fact, in one speed test I ran that was full of large matrix operations, PDL was the fastest.

But others, like Tela and Yorick, maintained speed even in scripted loops. Most other languages handle loops at much decreased speed. I also consider speed to be a factor that should include programmer time. Some languages use features seemingly easy to learn and allow coding to be quickly done. Others, like PDL, take a bit of getting used to, and use more verbose expressions that take more coding time. My estimates take all of these factors into account, but weights computational speed the most.

I/O flexibility may be of little consequence to those looking for a single environment for doing most of their work. But if one intends to use one of these tools in an environment where data is often operated on with other products, then clearly math languages with more extensive and controllable I/O features will be useful. Some of these languages stick to small handful of file structures (like HDF or MATLAB file types) and only offer high level routines for this purpose. Others give the user more fundamental I/O control so that functions can be designed to handle most any kind of file structure.

All of these languages offer matrices as a data structure. Several go beyond 2 dimensions for that purpose. But for processing many kinds of data, it is useful to be able to manage multiple data types in some kind of record structure, associating perhaps a few different sized matrices, some scalars, and even some strings into single entities. Some of these languages offer such structures, others do not. The PDL offers a number of such elevated data structures.

And finally, I consider the handling of strings. Granted, these languages are primarily for working with numbers. But often these numbers come embedded in files that also contain non-numeric information, such as data labels. And in any professional environment, it's likely that the results obtained with any of these languages must be output in some format useful for reports. Some of these languages offer very little of this kind of support, and some -- like PDL with it's underlying perl -- are not only great for numerical processing, but string processing as well.

I hope you use this information as only a starting point, and keep in mind the subjective nature of the evaluations. But if you know what kind of computing environment you'll be in, perhaps this graphic will give you a direction to begin your exploration.

I invite you to read my more detailed reviews of these languages. Links to the reviews are on the side-panel menu.