Announcing: mmockito 0.1.0 - MATLAB Mock framework based on Java Mockito

Well, the thesis got finished, and successfully defended, but the promised open-sourcing somehow took a back seat to celebrating and travel. Nevertheless, it's never too late to rectify the situation! Here is a copy of the announcement mail; more content will hopefully follow on the blog shortly (and probably doubling as documentation for the project).

I would like to announce the first release of mmockito, which is an open-source Mock framework for MATLAB, inspired by Java mockito. The work is a product of my bachelor's thesis and to my knowledge no other mocking frameworks exist for MATLAB. The project can be freely downloaded from its Github page:

https://github.com/vperic/mmockito

which can also be used to submit pull requests and report issues. A mailing list is also available for the project at:

mmockito@googlegroups.com

Documentation is (surprisingly) sparse at the moment; some is available through MATLAB and more information and a detailed comparison to Mockito and mockito-python can be found in my thesis, which can be freely downloaded. If you are interested in following the project, my personal blog will be updated occasionally; general discussion should be directed to the mailing list.

mmockito: How stubbing works

The previous post discussed some MATLAB-imposed limitations on the syntax we can offer. In the end, we've opted for a mock.when.aFunc(args).thenReturn(res) syntax. In the meanwhile, stubbing support has slowly matured and is essentially complete now, so it might be a good idea to showcase some of the functionality supported. While the syntax and the features provided were mostly inspired by (Java) mockito and python-mockito, we've diverged when we felt it made more sense for MATLAB code.

Basic usage

>> m = Mock();
>> m.when.aFunc(1,2).thenReturn(42);
>> m.aFunc(1,2)
ans =
    42
>> m.when.aFunc(1,'str').thenThrow(MException('a:b:c', 'exception!'));
>> m.aFunc(1,'str')
Error using Mock/subsref (line 57)
exception!

In the simplest use case, we can define an arbitrary function, provide the exact arguments it should take and the result it should give. "thenThrow" is offered as a convenience method to make code easier to read, the error would also be thrown if we just pass a MException object to "thenReturn". Also available is "thenPass", which is short for "thenReturn(true)".

Matchers

Because defining the exact arguments can be tedious, we've implemented Matchers. Matchers allow us to accept any argument satisfying a certain condition, instead of an exact value. Combining matchers and exact values is of course, also possible. For example, the Any matcher, if constructed with a class (either as a string or a meta.class object) only accept arguments of that class. If called with no arguments, the Any() matcher will accept anything at all.

>> m.when.a(Any('char')).thenReturn('ok');
>> m.when.a(Any(?double)).thenReturn('bad');
>> m.a('asdf')
ans =
ok
>> m.a(15)
ans =
bad
>> m.when.b(NumberBetween(1,5)).thenReturn(true);
>> m.b(3)
ans =
     1

It is very easy to define a custom matcher of arbitrary complexity, a couple of simple examples are included as a demonstration (and there might be a future blog post on this topic). A special case is the "ArgThat" matcher, which takes a "Constraint" in it's constructor (from the matlab.unittest.Constraints package). Since Constraints from the new unittest module serve basically the same purpose, they can be reused as Matchers. Our initial solution had us using Constraints directly, but they have a somewhat unfortunate naming scheme which doesn't "read" well. Nevertheless, with the "ArgThat" matcher users can reuse existing and custom Constraints, if needed.

>> import matlab.unittest.constraints.*;
>> m.when.a(ArgThat(HasNaN)).thenReturn('a Nan');
>> m.when.a(ArgThat(IsFinite)).thenReturn('no NaN');
>> m.a([5 6 14])
ans =
no NaN
>> m.a([1 2 NaN])
ans =
a Nan

Consecutive calls

Sometimes, we want to only stub a certain amount of calls. To do this, we can just chain the then* statements in any order. A "times" keyword is also offered for ease of use. Usually, the last stubbed return value will remain valid, but if we end the chain with the "times" keyword, it also will only apply the given number of times.

>> m.when.stub(1).thenReturn(1).times(2).thenReturn(2).thenReturn(3);
>> m.stub(1)
ans =
     1
>> m.stub(1)
ans =
     1
>> m.stub(1)
ans =
     2
>> m.stub(1)
ans =
     3
>> m.stub(1)
ans =
     3

It is important to note that the first stubbing is the one that is valid, and if it is setup for infinite stubbing it will not be possible to override it (short of creating a new mock). While in interactive use this might be a slight inconvenience, we get the added flexibility when making tests. For example, using Matchers, we can stub more complex behavior sanely.

>> m.when.stub(5).thenReturn('ok');
>> m.when.stub(Any(?double)).thenReturn('not implemented');
>> m.when.stub(Any()).thenReturn('bad!');
>> m.stub(5)
ans =
ok
>> m.stub(6)
ans =
not implemented
>> m.stub('asdf')
ans =
bad!

Strict vs. tolerant mocks; mocking real objects

By default, mocks are 'strict' -- methods which are not explicitly mocked will throw an error, the same as when calling a non-existent method. However, we can also create 'tolerant' mocks, which instead just return an empty list (MATLAB's equivalent of None or null in other languages).

It is also possible to pass a real object to the constructor of a Mock. In that case, if we are not stubbing a given method, we will pass it on to the object used in the constructor. Stubbed methods are always preferred, even if they shadow an existing method. Note: this feature isn't strictly complete as of right now and might change, but probably won't.


I hope that the presented features are compelling enough to interested some of you in using mmockito. If there's something more you'd want to see, don't hesitate to let me know. And, if you're interested in testing this code, I would be very glad for the feedback - please contact me! Coming up next, verification!

Mocking syntax and MATLAB's method handling shenanigans

One of the first decisions made when starting my thesis was to emulate the python-mockito syntax and provide similar features (namely, stubbing and verification). Why Python and not the original and more popular Java version, I'm not sure. Perhaps simply because when I approached my mentor, I was inquiring about any possible Python projects. In any case, the stubbing syntax boils down to:

when(mock).aFunction('arg').thenReturn('res')

Which actually looks quite clear and intuitive. Now, it was obvious from the start that to support such arbitrary calls to objects, I would have to overload the subsref method in MATLAB, and my initial prototype code did just that. I also knew that obj.method and method(obj) produce the same result in MATLAB, so my code handled only the first variant (it was a feasibility prototype, after all). This turned out to be a mistake - as far as I can see (and others agreed) - calling method(obj) does not, in fact, pass through subsref. As such, the above snippet is just not possible in MATLAB, erroring with: "Undefined variable "when" or class "when"."

But the error did give me an idea? What if I created a class called "when"? It could catch such calls and pass them on to the mock object, whose overloaded subsref could then handle them appropriately. In fact, it could even simplify it and that can only be a good thing. Sure, there would be the extra overhead of creating another class (no, a function wouldn't work), but these are supposed to be used in tests, which are hardly performance-critical (or rather, the added overhead can safely be ignored). Several lines of code later, everything worked great, the above line was correctly caught and I was just writing a script to make my results reproducible, when MATLAB backhanded me with another error:

Static method or constructor invocations cannot be indexed.
Do not follow the call to the static method or constructor with
any additional indexing or dot references.

The error message is clear enough - no indexing after constructors - but the code worked fine from the command prompt. Of course, there's already a StackOverflow question on this: it works in the command prompt and functions, but not in scripts. It even works in cell-mode, just not regular scripts! The answerer on SO seems to think this shouldn't work at all, but I disagree, and another blog gives a decent reason: using Java code in MATLAB. Someone suggested using "eval" to evaluate the code, but that is just not an option in this scenario, as it vastly decreases readability. As such, I've reached an impasse: directly working with the mock object doesn't seem to be a good idea, but creating a "fake" class for some reason doesn't work in scripts.

One line of reasoning is that mocks probably shouldn't be used in scripts (and cell-mode works, anyway) but mostly functions, so using the "when" class isn't a big problem. The issue, of course, is that sooner or later you're going to hit that very cryptic error message. A cleaner solution would be to offer a more verbose syntax for the mock object, perhaps something along the lines of:

mock.when('aFunction').with('arg').thenReturn('res')

Verbose, doesn't flow so well ("doesn't read like a sentance"), not to mention that it's a departure from other frameworks and mockito itself; also not an ideal solution. The third variant is to offer both: mock objects would expose the above API, and the "when" class would just call the mock with the appropriate syntax. But two ways to get the same thing is not good design. At this point I turn to you, dear readers, as I know I've managed to accrue at least a few interested followers. What would you like to see?