tag:blogger.com,1999:blog-88146965915611723922024-02-08T17:01:09.047+01:00Vlada's technical blogUnknownnoreply@blogger.comBlogger18125tag:blogger.com,1999:blog-8814696591561172392.post-38763181060353921432013-07-30T23:50:00.000+02:002013-07-30T23:50:03.835+02:00Announcing: mmockito 0.1.0 - MATLAB Mock framework based on Java Mockito<div dir="ltr" style="text-align: left;" trbidi="on">
Well, the thesis got finished, and successfully defended, but the promised open-sourcing somehow took a back seat to celebrating and travel. Nevertheless, it's never too late to rectify the situation! Here is a copy of the announcement mail; more content will hopefully follow on the blog shortly (and probably doubling as documentation for the project).<br />
<br />I would like to announce the first release of
mmockito, which is an open-source Mock framework for MATLAB, inspired
by <a href="https://code.google.com/p/mockito/">Java mockito.</a> The work is a product of my bachelor's thesis and
to my knowledge no other mocking frameworks exist for MATLAB. The
project can be freely downloaded from <a href="https://github.com/vperic/mmockito">its Github page</a>:<br />
<br /><a href="https://github.com/vperic/mmockito" target="_blank">https://github.com/vperic/<wbr></wbr>mmockito</a><br /><br />which can also be used to submit pull requests and report issues. A <a href="mailto:mmockito@googlegroups.com">mailing list</a> is also available for the project at:<br />
<br />
<a href="mailto:mmockito@googlegroups.com" target="_blank">mmockito@googlegroups.com</a><br /><br />Documentation
is (surprisingly) sparse at the moment; some is available through
MATLAB and more information and a detailed comparison to Mockito and
mockito-python can be found in my thesis, which can be <a href="http://cyber.felk.cvut.cz/research/theses/detail.phtml?id=407">freely downloaded</a>. If you are interested in following the project, my personal
blog will be updated occasionally; general discussion should be
directed to the mailing list. <br />
</div>
Unknownnoreply@blogger.com4tag:blogger.com,1999:blog-8814696591561172392.post-81741637050594584002013-04-22T16:55:00.000+02:002013-04-22T16:55:30.237+02:00mmockito: How stubbing worksThe <a href="http://www.vperic.blogspot.cz/2013/03/mocking-syntax-and-matlabs-method.html">previous post</a> discussed some MATLAB-imposed limitations on the syntax we can offer. In the end, we've opted for a <span style="font-family: "Courier New",Courier,monospace;">mock.when.aFunc(args).thenReturn(res)</span> syntax. In the meanwhile, stubbing support has slowly matured and is essentially complete now, so it might be a good idea to showcase some of the functionality supported. While the syntax and the features provided were mostly inspired by <a href="http://docs.mockito.googlecode.com/hg/latest/org/mockito/Mockito.html">(Java) mockito</a> and <a href="https://code.google.com/p/mockito-python/#Basics">python-mockito</a>, we've diverged when we felt it made more sense for MATLAB code.<br />
<br />
<h3>
Basic usage</h3>
<br />
<span style="font-family: "Courier New", Courier, monospace;">>> m = Mock();<br />>> m.when.aFunc(1,2).thenReturn(42);<br />>> m.aFunc(1,2)<br />ans =<br /> 42<br />>> m.when.aFunc(1,'str').thenThrow(MException('a:b:c', 'exception!'));<br />>> m.aFunc(1,'str')<br />Error using Mock/subsref (line 57)<br />exception!</span><br />
<br />
<span style="font-family: inherit;">In the simplest use case, we can define an arbitrary function, provide the exact arguments </span>it should take and the result it should give. "thenThrow" is offered as a convenience method to make code easier to read, the error would also be thrown if we just pass a MException object to "thenReturn". Also available is "thenPass", which is short for "thenReturn(true)". <br />
<br />
<h3>
Matchers </h3>
<br />
<span style="font-family: inherit;">Because defining the exact arguments can be tedious, we've implemented Matchers. </span>Matchers allow us to accept any argument satisfying a certain condition, instead of an exact value. Combining matchers and exact values is of course, also possible. For example, the Any matcher, if constructed with a class (either as a string or a meta.class object) only accept arguments of that class. If called with no arguments, the Any() matcher will accept anything at all.<br />
<br />
<span style="font-family: "Courier New", Courier, monospace;">>> m.when.a(Any('char')).thenReturn('ok');<br />>> m.when.a(Any(?double)).thenReturn('bad');<br />>> m.a('asdf')<br />ans =<br />ok<br />>> m.a(15)<br />ans =<br />bad<br />>> m.when.b(NumberBetween(1,5)).thenReturn(true);<br />>> m.b(3)<br />ans =<br /> 1</span><br />
<br />
<span style="font-family: inherit;">It is very easy to define a custom matcher of arbitrary complexity, a couple of simple examples are included as a demonstration (and there might be a future blog post on this topic). A special case is the "ArgThat" matcher, which takes a "Constraint" in it's constructor (from the matlab.unittest.Constraints package). Since Constraints from the new unittest module serve basically the same purpose, they can be reused as Matchers. Our initial solution had us using Constraints directly, but they have a somewhat unfortunate naming scheme which doesn't "read" well. Nevertheless, with the "ArgThat" matcher users can reuse existing and custom Constraints, if needed.</span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">>> import matlab.unittest.constraints.*;<br />>> m.when.a(ArgThat(HasNaN)).thenReturn('a Nan');<br />>> m.when.a(ArgThat(IsFinite)).thenReturn('no NaN');<br />>> m.a([5 6 14])<br />ans =<br />no NaN<br />>> m.a([1 2 NaN])<br />ans =<br />a Nan</span><br />
<br />
<h3>
Consecutive calls</h3>
<br />
Sometimes, we want to only stub a certain amount of calls. To do this, we can just chain the then* statements in any order. A "times" keyword is also offered for ease of use. Usually, the last stubbed return value will remain valid, but if we end the chain with the "times" keyword, it also will only apply the given number of times.<br />
<br />
<span style="font-family: "Courier New", Courier, monospace;">>> m.when.stub(1).thenReturn(1).times(2).thenReturn(2).thenReturn(3);<br />>> m.stub(1)<br />ans =<br /> 1</span><br /><span style="font-family: "Courier New", Courier, monospace;"><span style="font-family: "Courier New", Courier, monospace;">>> m.stub(1)<br />ans =<br /> 1</span></span><br />
<span style="font-family: "Courier New", Courier, monospace;">>> m.stub(1)<br />ans =<br /> 2<br />>> m.stub(1)<br />ans =<br /> 3<br />>> m.stub(1)<br />ans =<br /> 3</span><br />
<br />
It is important to note that the first stubbing is the one that is valid, and if it is setup for infinite stubbing it will not be possible to override it (short of creating a new mock). While in interactive use this might be a slight inconvenience, we get the added flexibility when making tests. For example, using Matchers, we can stub more complex behavior sanely.<br />
<br />
<span style="font-family: "Courier New", Courier, monospace;">>> m.when.stub(5).thenReturn('ok');<br />>> m.when.stub(Any(?double)).thenReturn('not implemented');<br />>> m.when.stub(Any()).thenReturn('bad!');<br />>> m.stub(5)<br />ans =<br />ok<br />>> m.stub(6)<br />ans =<br />not implemented<br />>> m.stub('asdf')<br />ans =<br />bad!</span><br />
<br />
<h3>
<span style="font-family: inherit;">Strict vs. tolerant mocks; mocking real objects</span></h3>
<br />
<span style="font-family: inherit;">By </span>default, mocks are 'strict' -- methods which are not explicitly mocked will throw an error, the same as when calling a non-existent method. However, we can also create 'tolerant' mocks, which instead just return an empty list (MATLAB's equivalent of None or null in other languages). <br />
<br />
It is also possible to pass a real object to the constructor of a Mock. In that case, if we are not stubbing a given method, we will pass it on to the object used in the constructor. Stubbed methods are always preferred, even if they shadow an existing method. <b>Note: </b>this feature isn't strictly complete as of right now and <i>might</i> change, but probably won't.<br />
<br />
<br />
I hope that the presented features are compelling enough to interested some of you in using mmockito. If there's something more you'd want to see, don't hesitate to let me know. And, if you're interested in testing this code, I would be very glad for the feedback - please contact me! Coming up next, verification!Unknownnoreply@blogger.com3tag:blogger.com,1999:blog-8814696591561172392.post-90134846906567404932013-03-06T22:36:00.001+01:002013-03-06T22:36:53.129+01:00Mocking syntax and MATLAB's method handling shenanigansOne of the first decisions made when starting my thesis was to emulate <a href="http://alexgorbatchev.com/SyntaxHighlighter/manual/installation.html">the python-mockito syntax</a> and provide similar features (namely, stubbing and verification). Why Python and not the original and more popular Java version, I'm not sure. Perhaps simply because when I approached my mentor, I was inquiring about any possible Python projects. In any case, the stubbing syntax boils down to:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">when(mock).aFunction('arg').thenReturn('res')</span><br />
<br />
Which actually looks quite clear and intuitive. Now, it was obvious from the start that to support such arbitrary calls to objects, I would have to overload the <a href="http://www.mathworks.com/help/matlab/ref/subsref.html">subsref</a> method in MATLAB, and my initial prototype code did just that. I also knew that <span style="font-family: "Courier New",Courier,monospace;">obj.method</span> and <span style="font-family: "Courier New",Courier,monospace;">method(obj)</span> produce the same result in MATLAB, so my code handled only the first variant (it was a feasibility prototype, after all). This turned out to be a mistake - as far as I can see (and <a href="http://stackoverflow.com/questions/15212641/how-are-methodobj-calls-handled-in-matlab">others agreed</a>) - calling <span style="font-family: "Courier New",Courier,monospace;">method(obj)</span> does <b>not</b>, in fact, pass through subsref. As such, the above snippet is just not possible in MATLAB, erroring with: "Undefined variable "when" or class "when"."<br />
<br />
But the error did give me an idea? What if I created a class called "when"? It could catch such calls and pass them on to the mock object, whose overloaded subsref could then handle them appropriately. In fact, it could even simplify it and that can only be a good thing. Sure, there would be the extra overhead of creating another class (no, a function wouldn't work), but these are supposed to be used in tests, which are hardly performance-critical (or rather, the added overhead can safely be ignored). Several lines of code later, everything worked great, the above line was correctly caught and I was just writing a script to make my results reproducible, when MATLAB backhanded me with another error:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">Static method or constructor invocations cannot be indexed.<br />Do not follow the call to the static method or constructor with<br />any additional indexing or dot references.</span><br />
<br />
The error message is clear enough - no indexing after constructors - but the code worked fine from the command prompt. Of course, there's already a <a href="http://stackoverflow.com/questions/10440409/following-calls-to-static-methods-with-indexing-when-importing-classes">StackOverflow question on this</a>: it works in the command prompt and functions, but not in scripts. <i>It even works in cell-mode, just not regular scripts!</i> The answerer on SO seems to think this shouldn't work at all, but I disagree, and <a href="http://blodgett.doof.me.uk/2010/10/01/hateyhatey/">another blog</a> gives a decent reason: using Java code in MATLAB. Someone suggested using "eval" to evaluate the code, but that is just not an option in this scenario, as it vastly decreases readability. As such, I've reached an impasse: directly working with the mock object doesn't seem to be a good idea, but creating a "fake" class for some reason doesn't work in scripts.<br />
<br />
One line of reasoning is that mocks probably shouldn't be used in scripts (and cell-mode works, anyway) but mostly functions, so using the "when" class isn't a big problem. The issue, of course, is that sooner or later you're going to hit that very cryptic error message. A cleaner solution would be to offer a more verbose syntax for the mock object, perhaps something along the lines of:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">mock.when('aFunction').with('arg').thenReturn('res')</span><br />
<br />
Verbose, doesn't flow so well ("doesn't read like a sentance"), not to mention that it's a departure from other frameworks and mockito itself; also not an ideal solution. The third variant is to offer both: mock objects would expose the above API, and the "when" class would just call the mock with the appropriate syntax. But two ways to get the same thing is not good design. At this point I turn to you, dear readers, as I know I've
managed to accrue at least a few interested followers. What would you
like to see?Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-8814696591561172392.post-45428723184150264312012-11-28T17:03:00.001+01:002012-11-28T17:03:08.737+01:00Announcing my Bachelor's thesis: Mock library for MATLABAfter positive experiences blogging about my GSoC projects, I've decided to do the same for my Bachelor's thesis. I've always felt that writing helped me process and refine my thoughts, and I'm hoping to leverage this for my thesis. Therefore, I plan on writing regularly, even about failed approaches.<br />
<br />
As the title suggests, my thesis will be about creating a library for Mock objects for MATLAB. During talks with my mentor, we concluded that, while a unit-test library for MATLAB exists (<a href="http://www.mathworks.com/matlabcentral/fileexchange/22846-matlab-xunit-test-framework">xUnit</a>) there is no support for <a href="https://en.wikipedia.org/wiki/Mock_object">mock objects</a>. Mocks are very useful as they allow developers to write robust, correct tests faster, so their lack in MATLAB can be felt. As MATLAB supports<a href="http://www.mathworks.com/help/matlab/object-oriented-programming.html"> object-orientated programming</a>, we are reasonably certain that Mocks can also be implemented. Thus, the first part of my thesis is to conduct a feasibility study and develop a simple prototype as a proof-of-concept. To do <i>that</i>, I would need to define exactly which features I am looking for.<br />
<br />
Coming from a Python background, my first instinct was to check existing Python Mock libraries (from <a href="http://pycheesecake.org/wiki/PythonTestingToolsTaxonomy#MockTestingTools">this nice list</a>). Most of them are ports of existing mocking frameworks to Python, though there are varieties. What was more interesting to me is that there is actually quite a bit of confusion about terminology (in particular, the term "mock" is used to refer to objects of vastly varying complexity). Investigating this, I came across an excellent article by Martin Fowler, <a href="http://martinfowler.com/articles/mocksArentStubs.html">Mocks Aren't Stubs</a>. The article is a great read throughout, as it goes into quite a bit of detail on "mocking" and general design issues; it also made me realize that most approaches are quite complicated. Taking a step back, it's fair to assume that the "average" user of MATLAB isn't a dedicated CS specialist, but rather a researcher. As such, I believe ease of use should be valued the highest, otherwise no one would take the time to learn these tools.<br />
<br />
This approach led me to <a href="https://code.google.com/p/mockito/">mockito</a> (nice presentation on <a href="http://szczepiq.files.wordpress.com/2008/08/dont-give-up-on-mocking.pdf">differences between mockito and other frameworks</a>, slides 38-40 provide direct comparisons of code). <i>mockito</i> defines itself as a "Test Spy" framework, where a "spy" is in essence a stub which remembers which interactions were invoked on it. This is a very natural, light-weight approach which is easy to pick up and almost as powerful as more verbose frameworks. After discussing it with my mentor, we both agreed that basing the new framework on <i>mockito</i> is the right approach. The next course of action is to create a basic set of use-cases, based on existing <i>mockito</i> examples, and start implementing them. This first code will be strictly proof-of-concept.<br />
<br />
Finally, one of the first decisions we made was to make the final, resulting code public; development will also probably happen on Github. Philosophically speaking, I would prefer that any work I do be actually useful, and the idea behind creating a library is for many people to use it, and that cannot be accomplished in the confines of a single faculty or university. It would also be useful to have some projects which would like to use Mocks, so that the library can be tested (and designed!) with real code in mind. Here I would like to turn to the community - if someone has felt a need for a similar library, please send me your desired use-cases and any other comments and ideas you might have. They would be a tremendous help!Unknownnoreply@blogger.com3tag:blogger.com,1999:blog-8814696591561172392.post-7118259077005187482012-08-03T20:26:00.000+02:002012-08-03T20:26:42.916+02:00Bootstrapping Trial in Python 3<div dir="ltr" style="text-align: left;" trbidi="on">
Initially, I had tried an "extensive" approach to porting Twisted - picking a certain error and fixing it in every module. Unfortunately, as I've found out, this isn't very practical: not only is Twisted a large code base, it's also very old. While updating this crufty code might have been doable, Twisted also has a requirement that all changes to code need to be tested (and I think this is very nice!*). This has been enforced quite strictly in the last few years, but of course, the code using the really old, Python3-incompatible idioms, is the same code which has no tests. As such, to make any sort of substantial change I would also need to write tests. This proved to be a little too much, and Itamar suggested I consider a more "intensive" approach - fixing Twisted a module at a time, starting with the core.<br />
<br />
In this I also meandered slightly, but after discussing it with exarkun on IRC, we concluded it would be best to pick a file with tests, run it under Python 3 and fix the failures which arise. This is in line with Twisted's programming paradigm, <a href="http://twistedmatrix.com/documents/current/core/howto/trial.html">test-driven development</a>, and is a very comfortable way of working. The idea, of course, was to start with modules which have no dependencies on the rest of Twisted, and then work "down" the dependency tree as individual modules are ported. While this sounds ideal, I've encountered two problems: the minor one is that Twisted depends on itself a lot, and it's hard (although not impossible) to identify modules which do not use any others; the major issue is the test runner itself, Trial.<br />
<br />
Trial is Twisted's (quite versatile) framework for testing, based on the standard unittest module. In time, the TestCase class was completely rewritten (though in a compatible way) to support various features which make testing easier. Now, when importing a file in Python 3, it needs to be syntax-compatible with Python 3, but <i>all of it's imports need to be compatible too</i>. So now, each test subclasses from twisted.trial.unittest.TestCase and the twisted.trial.unittest module is very large and unfortunately imports a large chunk of Twisted itself (notably, twisted.internet.reactor, but also half the twisted.python package). Therefore, it's impossible for me to actually run the tests, as I need Trial and Trial needs other things and none of this is compatible with Python 3. I had tried writing a large patch to at least make Trial importable, but it was rejected (and for good reason, I now think). Obviously, the huge patchset would need to be broken into smaller tickets, but preferably in a logical way.<br />
<br />
Luckily, the solution came via the official unittest module - if I only change the test case to import from the official library, rather than from Trial, it will work! Then a simple ``python3.2 -m unittest twisted.test.test_whatever`` runs the tests. I have successfully used this method for several simpler files but I fear the low-hanging fruit are gone - as was to be expected, many test files <i>do</i> use functionality provided only by Trial's TestCase. I am still trying to "pick around" here and there, and have also submitted tickets which do not fix a specific module, but just a single issue (eg. <a href="http://twistedmatrix.com/trac/ticket/5627">removing __cmp__ in t.p.versions</a>, <a href="http://twistedmatrix.com/trac/ticket/5789">removing uses of UserDict</a>). It is clear, however, that this approach will not lead me to my immediate goal - running Trial itself under Python 3. <br />
<br />
And this is where I currently am: my goal is to bootstrap Trial, to make it runnable in Python 3, which will make running tests (and, by extension, fixing relevant failures) much easier. The "pick a test_file and fix it" method cannot bring me there and I've been unable to think of a better alternative. One idea was to use an alternative TestCase implementation (where I tried testtools, which unfortunately isn't as-is compatible with Twisted's tests); using a different runner wouldn't help, as the modules would still need to be imported. Another idea is to provide some sort of temporary class, which would extend unittest from the official library with the specific methods I'm lacking; this class would then be deleted as soon as it's possible to run Trial itself. This doesn't strike me as a very clean approach, but it might be the only plausible one, unless someone has a different suggestion...<br />
<br />
In the meantime, I'm focusing on fixing what I can (even if it doesn't directly lead to supporting Trial) and making more "general" changes, to lower the size of further patches (but there will be at least a couple of big ones, there's no avoiding it). In fact, I've been focusing on making tickets as small as possible, to ease review burden, though I've still got plenty awaiting review: any help on this front would also be very appreciated. I've also tried reviewing other tickets, to ease the general burden, though the one case where I actually "passed" a review I had to revert the change, so I'm trying to be more careful about it now. <br />
<br />
<span style="font-size: x-small;">*While I do find it very nice, I do have some issues with this policy and I feel that a few carefully thought-out exceptions would have been very helpful in my project. More thoughts on this in a future blog post.</span></div>Unknownnoreply@blogger.com2tag:blogger.com,1999:blog-8814696591561172392.post-43962957300048064222012-06-18T12:26:00.000+02:002012-06-18T12:26:14.671+02:00Another year, another GSoC!<div dir="ltr" style="text-align: left;" trbidi="on">
Well, this blog post took long enough, but I'm happy to announce that I've been accepted once again for the Google Summer of Code, this year for <a href="http://twistedmatrix.com/trac/">Twisted</a> ("an event-driven networking engine written in Python"). What's more, my project is essentially the same as last year - porting Twisted to Python 3, or at least getting as close as possible (my actual proposal is available on <a href="https://docs.google.com/document/d/1N7PyYBrphVMLLgAR13fTXKsXjxD5m8HSkIXHpiPolKU" rel="nofollow">Google Docs</a>). Unfortunately, compared to last year, my school load was much higher this time around, so I've done much less work than I would have liked.<br />
<br />
At the start, I've mostly focused on fixing the warnings thrown when running the test suite with "-3", taking care of most of the trivial ones (eg. <a href="http://twistedmatrix.com/trac/ticket/5688">has_key</a>, <a href="http://twistedmatrix.com/trac/ticket/5707">apply</a>, <a href="http://twistedmatrix.com/trac/ticket/5689">classical division</a>). Currently, I'm looking into replacing "buffer()", which was a built-in but is removed in Python 3. While the work is similar, the workflow for getting changes in is quite different from SymPy. Twisted uses a svn repository, and trac for issue tracking; each change must be done in a separate branch and have a corresponding ticket; SymPy uses the classical Github + git workflow, with pull requests and reviews in the online interface. Now, I got too used to git to just give it up easily (especially as the Twisted workflow almost requires<a href="http://twistedmatrix.com/trac/wiki/Combinator"> additional tools</a> on top of vanilla svn), and <a href="http://twistedmatrix.com/trac/wiki/GitMirror">this guide</a> was very useful in setting up git svn. Although I'm getting used to the review process (eg. changesets are reviewed in total, not per-commit), I still find the Github (and git) model more productive - it streamlines review and allows small, atomic commits to be made (although I've been trying to keep my changes as small as possible, each such small change requires the opening of another ticket and creation of another SVN branch, so there's a point at which it's too much effort to do it all). Still, Twisted is unlikely to change so I will have to accommodate - at least I can follow my own practices in my own repo.<br />
<br />
The next steps are deciding which porting strategy to ultimately pursue - some Twisted developers suggest a single code-base strategy (py2 and py3 compatible), I personally favor a single code base which relies on 2to3 for py3k compatibility and there was even <a href="https://bitbucket.org/pitrou/t3k/wiki/Home">an attempt at a dual code-base</a>, by Antoine Pitrou. While I feel that approach is the least likely to succeed, as it introduces a high burden on the maintainer (and the effort has indeed stalled), the code already there will be helpful in my own work. Often, the changes made in py3k code can be reused in the "main", Python 2 code with little or no changes. Still, all approaches deserve investigation and my mind is still open to other ideas. Twisted currently supports Python 2.6+, which makes my job easier. The final piece of good news is that I may be able to get some sort of help (or at least support) from Canonical, as part of their <a href="http://www.wefearchange.org/2012/04/python-3-on-desktop-for-quantal-quetzal.html">plan to install only Python 3 in the next desktop Ubuntu release</a>.</div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8814696591561172392.post-48296318527425792312012-02-10T17:27:00.000+01:002012-02-10T17:27:46.004+01:00Thoughts on Google Code-in 2011Google Code-in is the high-school equivalent of the Google Summer of Code. The program ran from Nov 21st to Jan 16th, though we've only now gotten around to sending <a href="https://groups.google.com/group/sympy/browse_thread/thread/fde434aa771fd5eb/c3220755357b032a">a "summary" mail to the list about it</a>. As Aaron noted, we've had some translation work, some work on SymPy Live and a bevy of documentation and code improvements. With 176 tasks completed, I'd say the whole project was a success for SymPy. I was involved as a mentor, so here are some general thoughts and observations about the process.
<br /> <br />
<b>E-mail spam.</b> In SymPy we didn't have a clean separation of mentor duties (eg. KDE only allowed tasks for which someone volunteered to mentor), so the initial idea was to add all (most) mentors to all tasks. This meant a lot of mails, an effect worsened by the fact that each commenter to the issue starts another "conversation" when viewed from Gmail (which I even <a href="https://code.google.com/p/soc/issues/detail?id=1388">reported to Melange</a> as a feature request/bug). At the height of activity, I could get upwards of 30-40 mails ("conversations") daily, which by far dwarfed my other mail traffic. Then, because each comment is basically a separate mail, I wasted a lot of time looking at issue that someone already addressed (again, most mentors could handle most tasks). For the second round of tasks I didn't add myself to each task, otherwise I'm sure I'd have gotten even more spam. The bug I reported in Melange was fixed, so hopefully this will be less of an issue next year.
<br /> <br />
<b>Being a mentor takes a lot of time.</b> Partly a consequence of above, partly due to all the work being done, but being a mentor took a lot of time. Many students were unfamiliar with git (and didn't want to read the instructions on development workflow on our excellently-written (in my opinion) <a href="https://github.com/sympy/sympy/wiki/GCI-2011-Landing">GCI Landing Page</a>) and solving issues with them was a constant topic on IRC. Students also lacked follow-through with comments (or, occasionally, expected the work handed down to them) which didn't help. Finally, many students were very anxious, and didn't appreciate that we are all volunteers and cannot be around 24/7. All of this resulted in a process that was frustrating at times and stressful for mentors.
<br /> <br />
Regardless of all of the above, <b>a lot of work was done</b> for SymPy. While I didn't look at the stats, my feeling is that the biggest improvement could be seen in our SymPy Live interface (and our webpage) and our documentation. Yes, we also saw some code improvements, but they were probably a smaller part of the overall contribution (though by no means less important). Interestingly, I think this exposes the two types of tasks the GCI contest is well-suited to: tasks where there is no "in-house" expertise (anything web related in our case) and uninteresting tasks/chores (writing documentation, in our case and probably for most projects). In the first case, we managed to attract experienced developers who could improve our webpage much faster and better than any of the core developers. Writing documentation is also an important task, but one that is shunned by most developers. Still, it is mostly simple work and (more importantly) doesn't usually require in-depth understanding of the code. This made it ideally suited for new contributors. The financial award (100$ for every 3 completed tasks, up to 500$) was enough of a motivation for students. The all-around improvements to our documentation are probably the single biggest advantage of our participation in GCI.
<br /> <br />
<b>Translations.</b> In GCI, tasks were divided into categories and we needed to have at least 5 tasks in every category. While we managed to "fill-up" most categories, Translation was probably the biggest problem. As a, basically, command-line library, it does not make a lot of sense for SymPy to be translated in other languages. In the end, we created tasks for translating our webpage and tutorial to the languages covered by the development team and some of these were done, but I consider this a waste of time. Though this issue is "near and dear" to me (I'm not a native speaker of English), I'm of the opinion that it would be impossible for someone without at least a basic knowledge of English to program with SymPy. Simply, however much effort we put into translating, the class and method names will remain in English and there's no helping that. I very much doubt the newly translated documents will be even used and they're bound to fall behind as the original document changes. We also had to start using gettext to manage the translations, which is a non-trivial amount of work (and there are still some issues). In my opinion, it adds another layer of complexity (however small) for very little gain.
<br /> <br />
In <b>conclusion</b>: did we get stuff done? Yes, without a doubt. Would we have gotten more if the mentors used their mentoring time for coding? Perhaps, but not necessarily. Are some of the students going to keep contributing? Most likely not. Still, I would consider the whole program, and our participation in it, a <b>success</b>. Ideas for next year could be focusing more on stuff none of the core developers can do (eg. the website work), but we can't really say how far along will SymPy development progress during this year or which tasks might be available to students. Hopefully, more people will volunteer to mentor next year, which would help with most issues I raised here. It is interesting, though, that even with our normally very fast development process we couldn't handle the influx of student work. It'd be interesting to see how other organizations coped.
<br /> <br />
Here's to another GCI this year!Unknownnoreply@blogger.com2tag:blogger.com,1999:blog-8814696591561172392.post-73966574870354703472011-11-01T00:02:00.000+01:002011-11-01T00:02:24.179+01:00GSoC: Final Report<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-size: large;"><b>Introduction</b></span><br />
<br />
My project, officially named just "Porting to Python 3", is actually divided into two separate (albeit mutually complimentary) parts: my first goal was to set up a testing framework, to ensure continuous testing of SymPy across different versions of Python. SymPy used to have a server that ran buildbot, but it stopped working some time ago. This was to provide a solid base for working on my main project, making the code Python 3 compatible. As this was to be done with a single code-base, I estimated (correctly) that these could lead to subtle incompatibilities between various Python versions. Time permitting, I also intended to leverage this framework and my knowledge to get SymPy compatible with PyPy, too. The original application can be seen at the SymPy Wiki, <a href="https://github.com/sympy/sympy/wiki/GSoC-2011-Application%3A-Vladimir-Peri%C4%87%3A-Porting-to-Python-3">here</a>. More details about my progress can also be found in <a href="http://vperic.blogspot.com/">my blog</a>.<br />
<br />
<span style="font-size: large;"><b>Testing framework</b></span><br />
<br />
As mentioned above, SymPy used to use buildbot, so this was my first choice. I also read about Tox, a tool that is written for the sole purpose of testing Python programs under various conditions (different interpreters, presence or lack of certain dependencies), which also provides good integration with Jenkins, a well known server for continuous integration. My initial thoughts on this are in <a href="http://vperic.blogspot.com/2011/05/continuous-integration-and-sympy.html">one of my first blog posts</a>, where I had decided to use Tox and then later try to integrate it with Jenkins, to form a fully functioning CI server. While Tox was immediately useful (here's <a href="http://vperic.blogspot.com/2011/06/setting-up-and-using-tox-in-sympy.html">a post on setting up and using it</a>), the integration with Jenkins proved to be more arduous than my initial tests showed. In retrospect, perhaps I should have given buildbot a more thorough look later, rather than eliminate it so early. Furthermore, while I saw Tox as a great tool, the uptake among other developers has been.. less than stellar (other than Aaron, I'm not aware of anyone using it regularly).<br />
<br />
Fortunately, parallel to me setting up Tox/Jenkins, work was progressing on <a href="https://github.com/sympy/sympy-bot">sympy-bot</a>. The main need for continuous integration came from a desire to review all pull requests and test them for errors - while bigger companies and projects might need real CI, all of SymPy's code gets in through the GitHub pull request system, so theoretically it should be enough to just thoroughly test every pull request; sympy-bot was developed with this purpose in mind. Designed to be ran manually, it still has the basic functionality which I couldn't manage to replicate in Jenkins: run the test suite and post the results back. Work on it has also quickened somewhat in the last couple of months, and <i>I now consider further development of sympy-bot a better idea than working more with Jenkins.</i><br />
<br />
<span style="font-size: large;"><b>Python 3 porting</b></span><br />
<br />
Even with the relative failure of setting up a robust testing framework, my main project was also progressing. Due to the nature of the issue, progress was somewhat sporadic and didn't proceed at a steady pace. This was particularly apparent during the start - I was simply stumped by some of the errors I was getting and couldn't get around them; once I made a key breakthrough, I was quickly able to get SymPy importable under Python 3, though this only happened <a href="http://vperic.blogspot.com/2011/06/gsoc-week-5-sympy-now-runs-in-python-3.html">by week five</a>. The rest of my summer was spent hunting down the remaining errors, which was interesting at first but got very tiresome by the end. In fact, at the end Mateusz had to step in and fix the remaining few failures as I simply couldn't bring myself to look at them <i>yet again</i>. Thanks Mateusz! [Mateusz also did a lot of work on improving PyPy support, something for which I simply didn't find the time, so double thanks to Mateusz!]<br />
<br />
One issue that arose early during the porting process was the (un)bundling of libraries with SymPy. SymPy bundled Pyglet and mpmath. Bundling the first was probably a bad idea at the start, and it was finally removed by Stefan Krastanov sometime early in the summer to unanimous approval. Unbundling mpmath was a more contentious issue, it sparked <a href="https://code.google.com/p/sympy/issues/detail?id=2482">a very lively discussion on the issue tracker</a>. I won't rephrase it here, but in the end it was decided <b>not</b> to unbudle it. This meant that I had to write a custom tool to handle calling 2to3: we needed to avoid calling it on the mpmath/ directories, because mpmath is already py3k compatible (and running 2to3 on such code produces bad code).<br />
<br />
It was ultimately decided that this tool will live in bin/use2to3 and work by creating a Python 3-compatible version of the source code in a py3k-sympy/ subdirectory (originally sympy-py3k/ but that interfered too much with tab-completion!), from which SymPy could then be ran normally under Python 3. While I initially had misgivings about the script, I now think it's quite powerful. It's not the most ideal solution, but it does work and was the last missing link in seamless Python 3 support (eg. it also corrects shebangs and fixes some whitespace issues caused by 2to3).<br />
<br />
<b><span style="font-size: large;">Conclusion</span></b><br />
<br />
Officially, my project was a success, but I really couldn't have done it without the help of other developers working on SymPy, in particular Ronan, Aaron and Mateusz. Beyond the GSoC period, I've got every intention to continue working with SymPy, as I think I've already shown with the few pull requests I've submitted since; I have also decided to take a more active role in helping with the <a href="https://github.com/sympy/sympy/wiki/GCI-2011-Organization-Application">Google Code-In project</a> (assuming SymPy is accepted). As for my project, I intend to focus more on the infrastructure needed to support SymPy, rather than the math issues. Still, as my knowledge of math and SymPy internals increases, I'm sure I'll find other places to contribute as well.<br />
<br />
To future GSoC students, I suggest maintaining good communication links and trying to be involved with the project as much as possible. Good communication with the core developers and general awareness of the current state of SymPy helped me a lot. While this was arguably more important for my project than others, at least <a href="https://github.com/sympy/sympy/wiki/GSoC-2011-Report-Sean-Vig-Symbolic-Clebsch-Gordon-coefficients%3AWigner-symbols-and-Implementing-Addition-of-Spin-Angular-Momenta">Sean Vig</a> has also expressed regret at not being more involved. The second most important bit of advice is to try and split your work into multiple pull requests and try to get them merged as fast as possible. SymPy has a very rapid pace of development, and as such it is <i>always</i> better to integrate sooner rather than later. This ties in to making good, atomic commits, but means more than that: your work should be clearly separated into small, logical chunks (<= 20 commits is my suggestion). A lot of the work done this summer has still to be integrated, or there were many troubles getting it finally in (eg. the physics.mechanics module). Finally, try to budget a lot of extra time in your project application - most of us are not experienced developers and cannot estimate the amount of work needed for something correctly. Plus, when some additional problems arise (and they <i>will</i>), it's always better to have time set aside to deal with them.<br />
</div>Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-8814696591561172392.post-38594829344700160152011-08-19T21:39:00.000+02:002011-08-19T21:39:38.271+02:00GSoC: final weekWell, I obviously haven't kept up with my blogging. Partly this is because there wasn't too much to report - I've been stuck at the few remaining bugs for a couple of weeks now and just couldn't make myself go at them for real. Still, I've now tentatively submitted <a href="https://github.com/sympy/sympy/pull/558">a "final" pull request</a>, which has some misc fixes and more importantly adds the "use2to3" script that will form the basis of our Python 3 support. The idea is to run the script, which will create a "sympy-py3k" directory, which is a copy of the SymPy directory structure but with 2to3 ran where required. It should then be possible to use SymPy normally under Python 3 from that folder (even installing it). Now, this script could do with some improvements (looking at "git ls-files" instead of traversing the whole dir structure for one), but most importantly it is <i>not really well tested</i>. I've only tested it on my computer and while it should be sound there have already been some problems reported (in the pull request). Still, the script should hopefully be robust and so, if you have the time and/or care about Python 3 support at all: <i>please</i> try out the script and see if it breaks. You can get it in my "porting4" branch.<br />
<br />
Now, there are some issues remaining (Ronan is working on <a href="http://code.google.com/p/sympy/issues/detail?id=2574">the LambertW issue</a> and the other two, <a href="http://code.google.com/p/sympy/issues/detail?id=2609">test_priority error</a> and <a href="http://code.google.com/p/sympy/issues/detail?id=2611">tensor doctest failures</a> are on the issue tracker), but I still consider my project as successful. Python 3 support is a moving target anyway (all new code that gets in is another potential error, at least until people start actively testing it) and I also expect issues to crop up that just aren't covered by the test suite. For the moment, Python 3 support can be considered an experimental, dev feature and I'd like to keep it that way for a few weeks before considering a release. Again, if you are interested in Python 3 support, please try it out, start using it for your normal work and note if something unexpected comes up. <br />
<br />
For the foreseeable future, I definitely intend to stay with SymPy. I found the hacking genuinely interesting and would like to finish my stated goals of achieving both Python 3 and PyPy compatibility. Part of my reasoning is completely selfish - I expect these skills to be quite valuable in the near future and hacking one a code base as large as SymPy is bound to teach me a few tricks. Mostly, though, I'd just like to make SymPy better - my line of thought being, if I can do the mundane infrastructure work, then that will enable the mathematicians and physicists among us to implement more cool algorithms. <br />
<br />
I'd also like to extend my thanks to Ronan Lamy, my mentor, who was kind enough to solve a few issues for me and for generally always being around to point me in the right direction; to Aaron Meurer, the maintainer of SymPy, who always had the time for a detailed review of my code, even if our views didn't always coincide :) (<a href="http://code.google.com/p/sympy/issues/detail?id=2482">mpmath, anyone?</a>); and finally, to all the other devs in SymPy who've helped me by reviewing my code and for making such a great program in the first place! Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8814696591561172392.post-31325467789748063042011-07-25T15:47:00.000+02:002011-07-25T15:47:56.909+02:00GSoC: week 9: Almost thereI've just now seen I managed to completely miss my blog post last week. Bah! So, for the past two weeks, I've worked on two things in parallel. Lets take it from the top:<br />
<br />
<span style="font-size: large;">Python 3 support</span><br />
<br />
This is going as well as could be hoped. <a href="https://github.com/vperic/sympy/tree/porting">The latest pull request</a> is awaiting merging. With it, there are three exceptions and one failure in the main test suite remaining and this hasn't changed in about a week. I've asked Ronan for help on these, and he's opened <a href="http://code.google.com/p/sympy/issues/detail?id=2574">an issue</a> for one of the exceptions. It's something deep in the assumptions code; as Ronan knows his way best around this code, I hope he'll be able to solve it. I haven't looked in detail at the other problems, but hopefully they won't be too hard.<br />
<br />
What I have been doing is working on the doctests and the failures they show - at least three real issues that aren't covered in normal tests have beeen uncovered. The others have been mostly problems with the doctests themselves - doctests rely on comparing the exact output and some minor things have changed in Python 3 (eg. <type 'int'> is now <class 'int'>). Most of these fixes are in the pull request referenced above. There are 2 exceptions and 2 failures remaining to fix. One of the failures (while trivial to hack-fix) actually exposed <a href="http://code.google.com/p/sympy/issues/detail?id=2590">issue 2590</a>: "jn_zeros in functions/special/bessel.py should return SymPy Floats, not Python floats". The rest shouldn't be too bad (though I wish there was someone around to help me with tensor/index_methods.py). <br />
<br />
The real problem was getting the doctests to actually run. Now, to start with, doctests also need to be converted by 2to3. This isn't done automatically but by passing a "-d" flag to 2to3 (which converts only doctests). Unfortunately, this crashed in ntheory/factor_.py because of the use of reduce(). I couldn't work around this so I've opened <a href="http://bugs.python.org/issue12611">an issue</a> upstream (I've also opened <a href="http://bugs.python.org/issue12616">two</a> <a href="http://bugs.python.org/issue12613">more</a> issues for other 2to3 deficiencies that I've managed to work around). This is a problem because even if there's a fix upstream it will only apply to new Pythons. For the moment I've just deleted the relevant bit of doctest, but this warrants further discussion. [EDIT: In fact, I've just opened <a href="http://code.google.com/p/sympy/issues/detail?id=2605">issue 2605</a> for this] There's one more fix required (unicode-related) but that can just be wrapped with a version check.<br />
<br />
The second problem was running the .txt doctests. First I learned that I need to pass the .txt files to 2to3 explicitly, they aren't caught otherwise (this from Lennart Regebro's <i>excellent</i> <a href="http://python3porting.com/index.html">Porting to Python 3</a> book, which is now available for free; thanks Lennart!). Then it turned out our doctest runner depends on an internal procedure (_load_testfile) which changed from Python 2 to Python 3 to have an extra argument for an encoding. Finally, the doctests had a few errors in them but nothing major (one polys issue, though). The real peculiar thing is that some lines seem to print an extra "None" after what they're supposed to print; this happens in tutorial.txt and matrices.txt but nowhere else. As I haven't been able to reproduce this, I must conclude it's related to our testrunner somehow. Furthermore, this only happens with .txt doctests which further narrows it down. It's not clear to me how to solve this yet. Our doctest runner is a bit of a mess, using slightly different methods for .py and .txt doctests and copy-pasting some upstream methods but not all. One solution might be copying the rest of the code over, another could be porting us to py.test. This is something to raise at the next (IRC) meeting with my mentor. It's a minor issue, not impacting any real functionality, but as it could hide other issues it should be fixed.<br />
<br />
We've also reached a solution - of sorts - on the unbundling of mpmath. As Aaron and Ondřej were adamantly against unbundling it, and as no clear benefit was then seen to porting to Distribute (not that it'd be easy without unbundling), I've decided to "concede" that argument. At the moment, the plan is to write a script which would copy the code to a sympy-py3k directory and run 2to3 as appropriate on it. This will probably require changes to various files (setup.py at least should then be compatible with both Python versions without the need for 2to3) and it's not yet clear how all details will be handled but it's definitely possible. I feel such a solution is going to be fragile by definition and could lead to problems down the line, but it's also basically the only way to have my work visible to users in the short run so I will go along it.<br />
<br />
Nevertheless, I feel I'm very close to completing my goal (and the Python 3 part of my project) of porting SymPy to Python 3. Once the above script is decided upon and implemented, I hope to get wider testing from other developers and users. It is almost certain that there will be issues not caught by the tests and as I don't use SymPy myself having others test them is the only way for me to see them. There's an imminent minor release of SymPy, 0.7.1., which unfortunately doesn't include much of my work. I hope, however, that once it's fully complete and merged it will quickly be followed by a 0.7.2-alpha release to allow testing beyond the developer community.<br />
<br />
<span style="font-size: large;">Jenkins for SymPy</span><br />
<br />
Last week Ondřej was free to work on reinstalling the current server we use and I've since setup a Jenkins CI server there; you can find it at the <a href="http://72.14.182.119:8080/">old address</a>. As before, anonymous users can see the build history while developers should drop me a note if they'd like an account. Currently the builds still have to be triggered manually, but I hope to get the Github-integration complete and have tests run on every change pushed. The <a href="http://code.google.com/p/sympy/issues/detail?id=2585">crucial issue</a> here was that the tests were taking up too much memory, causing Jenkins to swap and take hours to run the test suite. This was fixed with <a href="https://github.com/sympy/sympy/pull/507">pull 507</a>, which clears the cache after running each file. This more than halved the memory usage (from ~450 MB to less than 200) which allows Jenkins to complete the 6 test runs (py25-27 x {python, gmpy} ground types) in just under an hour. The change should otherwise be harmless enough, but issues could arise (which is why it isn't included in the upcoming release).<br />
<br />
Increasing the number of tests ran (scipy and numpy integration should also be tested, but in separate jobs probably) and having a working framework for automatically testing pull requests are the remaining tasks, but I consider the Jenkins server running and this part of my project a success. It will also be invaluable when Python 3 support is finally in (as it's unlikely many, if any, developers will test on Python 3 when developing). I've also added another project to continually test a single file to help another GSoC student, <a href="http://nessgrh.wordpress.com/">Tom Bachmann</a> (ness), iron out some issues with random numbers in tests. It tests his branch every 5 minutes, which is many hundreads of runs daily and a lot more coverage than a single developer could reasonably provide. I consider it an experiment, to see how feasible it is to offer a level of extra support for critical changes. So far seems to be working fine. There's also a Jenkins plugin worth investigating, which implements priority for projects, which could allow us to have many such "side" experiments without impacting the main testing process whatsoever. The plugin is currently under development, but it is something to look into.<br />
<br />
<br />
Phew! That was one long post! Hopefully, it makes up for the missing last week post at least a bit.Unknownnoreply@blogger.com7tag:blogger.com,1999:blog-8814696591561172392.post-9866102588261612082011-07-11T14:21:00.000+02:002011-07-11T14:21:48.963+02:00GSoC: week 7Ack! This has been a really bad week for me! I'd like to blame it on the intermittent internet access, but the reality is that I've just come home after about 6 months and it's hard to avoid catching up "just a bit". I did manage to fix some of the unicode issues bringing us down to just two test failures and 9 exceptions, but I should've had it all working by now. In other news, Renato has worked some more on <a href="https://code.google.com/p/sympy/issues/detail?id=1026">running in PyPy</a>, bringing it down to one exception and 9 failures - it's quite possible that some of those are actually SymPy errors where we rely on some CPython-specific detail, those are going to be particularly interesting to track down.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8814696591561172392.post-32968010647832842982011-07-03T17:04:00.000+02:002011-07-03T17:04:44.208+02:00GSoC: week 6: Further Python 3 porting workContinuing on <a href="http://vperic.blogspot.com/2011/06/gsoc-week-5-sympy-now-runs-in-python-3.html">last week's progress</a>, I've worked on the other exceptions and, with the help of Mateusz and Ronan, we are now down to some 33 test failures and 9 exceptions. I've sent a pull request with my current work, you can see it <a href="https://github.com/sympy/sympy/pull/473">here</a>; I've agreed with my mentor that sending frequent pull requests is the right path to take. I've been traveling for a few days, but my immediate goal is to bring the above numbers down to zero and get the doctest suite to run (which should point out further errors). The pull request also contains a commit marking all errors with a "FIXME-py3k" in the code, which should hopefully encourage other developers to take a look.<br />
<br />
My other goal for next week is to port us to Distribute finally, which shouldn't be too hard. Like last week, the biggest hurdle is the decision on mpmath. Once that is taken care of, I'll see about using py.test again (SymPy currently uses an older fork we made) - this will make testing easier (as there have been quite some improvements since it was forked) and will also help when we finally start using Jenkins as it can export data to JUnitXML (which is handled well by Jenkins). This has the downside of disabling our benchmarking suite, but I've agreed with Ronan that we can keep the old code around until it too can be ported. It's hardly used, so it's not that big of a priority.<br />
<br />
Another piece of good news is that the core SymPy tests now run under PyPy, which is the second part of my GSoC project. You can see <a href="http://code.google.com/p/sympy/issues/detail?id=1026">the relevant issue</a> and star it to automatically get any updates. Thanks to Renato for trying this out and reporting back!Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8814696591561172392.post-12136912747342296592011-06-26T17:57:00.000+02:002011-06-26T17:57:02.030+02:00GSoC: week 5: SymPy now runs in Python 3!<div dir="ltr" style="text-align: left;" trbidi="on">As the title says, you can now run SymPy in Python 3! No, it's not perfect, there are test failures (or rather, exceptions, some 200 of them), but ~90% tests pass and that's getting somewhere. I haven't tried actually using it, but at least for some definitions of "works", SymPy definitely works in Python 3. I'll document some of the issues I solved below, and I'll also update the <a href="http://vperic.blogspot.com/p/python-3-porting-tips-and-tricks.html">porting tips and tricks</a> page when I get the chance.<br />
<br />
<span style="font-size: large;">The cmp issue</span><br />
<br />
While the new unicode/string features are probably the major Python 3 incompatibility for most programs, SymPy's foremost issue was the use of cmp(), which has been discontinued in Python 3 in favor of __lt__ and other rich comparisons. SymPy uses cmp() in various areas of code (sympy.core.Basic for one, and almost everything derives from that). I've tried to approach this from various angles, but most required a deeper understanding of SymPy than I currently posses. In the end, I opted for the most ingenious solution of all - sidestepping the issue. To see how I did this, first lets see the two different ways the removal of cmp() affects us:<br />
<br />
<b>a) <tt class="xref docutils literal"><span class="pre">builtin.sorted()</span></tt> and <tt class="xref docutils literal"><span class="pre">list.sort()</span></tt> no longer accept the <i>cmp</i> argument providing a comparison function. Use the <i>key</i> argument instead.</b><br />
<br />
Now, this is a good thing! The cmp() function is slow in this case, as it will compare every two elements (or so). In comparison, the key function takes some defined "key" value and sorts based on; this means it's called exactly once on each object. However, if you rely deeply on customized __cmp__ functions, it can be difficult to simply use a key instead of cmp. The <a href="http://code.activestate.com/recipes/576653-convert-a-cmp-function-to-a-key-function/">following recipe</a>, which provides a cmp_to_key function, allows us to avoid this by doing just what the name says. It's not an elegant solution, but it allows us to move on with porting at least. The recipe has been included in the functools module in Python's 2.7 and 3.2, but we implement it in core.compatibility as we need to support more Python versions.<br />
<br />
<b>b) The <tt class="xref docutils literal"><span class="pre">cmp()</span></tt> function should be treated as gone, and the <tt class="xref docutils literal"><span class="pre">__cmp__()</span></tt> special method is no longer supported.</b><br />
<br />
This is the root cause of the above point. Again, SymPy uses this quite extensively and fixing everything is beyond me at the moment. However, it is possible to naively define cmp as (a>b) - (a<b) if the builtin is not available, and I did exactly that in core.compatibility. That this works means that we aren't doing anything too complex with out __cmp__ implementations (it does fail in one case, though), which means that porting is possible.<b><br />
</b><br />
<br />
Neither of the above changes is efficient or very nice at all (in fact, I described them as "evil" in a commit message). Still, going over these issues allows me to see the other problems that will arise and work on fixing them. It will allow me to finally have some visible progress on my project. This is important because my goal for the mid-term evaluation is to have a working SymPy in Python 3. Once SymPy is working completely, I will go back and implement a "correct" solution for this.<br />
<br />
The other issues I solved are comparatively minor: I fixed some warnings produced by "-3" (the backquote is no longer a shortcut to repr(), tuple parameter unpacking has been removed) and some of the exceptions raised when running with Python3 (file() has been removed and we were encoding to ascii needlessly. I plan to continue working on these exceptions; as a lot of the tests all fail at the same points, I except to rapidly have almost all tests running in Python 3 (if not passing). I've also added a "FIXME-py3k" comment in a couple of files where I identified the issue but couldn't solve it; these are all things to discuss with my mentor. If someone wants to test it out, I've written some instructions on <a href="http://code.google.com/p/sympy/issues/detail?id=1262">the relevant issue</a> in our bug tracker. Merging will probably wait some time, but I will push for it to happen as soon as possible.<br />
<br />
<br />
Also during this week, I worked on setting up our Jenkins server, you can see it <a href="http://72.14.182.119:8080/">here</a>. Anonymous users can see just the build history; developers should contact me if they want an account set up. It is currently down, however.This was mostly a test run - we decided to reinstall our server from an ancient Debian to a current Ubuntu and only then run the "live" Jenkins server. This will happen when Ondřej, who runs the server we use, has more time (around the middle of July). I don't expect further issues to arise.<br />
<br />
I also have a test branch with a port to distribute (instead of the currently distutils we use). We need to use Distribute to support automatic running of 2to3 when installing. The porting itself was trivial, but I need to read a <i>lot</i> of documentation before I figured out what the differences are (I even asked <a href="http://stackoverflow.com/questions/6344076/differences-between-distribute-distutils-and-setuptools">a question on StackOverflow</a> about it). To say the documentation is confusing would be an understatement. The major problem is how to solve the mpmath issue.<br />
<br />
As I wrote in <a href="http://vperic.blogspot.com/2011/06/gsoc-week-3.html">my week 3 report</a>, mpmath is already Py3 compatible and 2to3 shouldn't be run on it. I've since checked it out and I don't see an easy method to skip the mpmath directory. It <i>might</i> be possible, but I don't see an easy way without going around Distribute entirely (which is, again, not the point). I even asked about in on the distutils-SIG mailing list, the answer is a "<a href="http://mail.python.org/pipermail/distutils-sig/2011-June/017901.html">don't do it, unbudle instead</a>". While I've been in the unbudle camp since the start, <a href="http://code.google.com/p/sympy/issues/detail?id=2482">the discussion</a> hasn't been so one-sided (I won't summarize it here, you can read the linked issue). In the end, though, if I don't find a way to support it in Python 3, we will have to unbundle (Ronan has <a href="https://github.com/rlamy/sympy/tree/unbundle-mpmath">a branch</a> doing just this, and it is fairly painless). </div>Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-8814696591561172392.post-14052524246049415332011-06-19T14:55:00.000+02:002011-06-19T14:55:30.926+02:00Setting up and using Tox in SymPy<div dir="ltr" style="text-align: left;" trbidi="on">In my previous post <a href="http://vperic.blogspot.com/2011/05/continuous-integration-and-sympy.html">on the subject</a> I mentioned we decided on using Tox. In the weeks since, we've established a workflow and successfully used Tox to find various bugs (see my post <a href="http://vperic.blogspot.com/2011/06/gsoc-week-3.html">from last week</a> for a particularly interesting one). As I've found the Tox documentation confusing in places, I've decided to write a "how-to" for setting up Tox and then talk some more about how we use it. <br />
<br />
<span style="font-size: large;">Setting up Tox</span><br />
<br />
A simple "pip install tox" will handle the installation. Then you'll want to create a tox.ini file where your setup.py is located. The .ini can be very simple, just listing the wanted environments and the commands to run in each.<br />
<blockquote><pre style="font-family: "Courier New",Courier,monospace;"><code>[tox]
envlist = py25, py26, py27
[testenv]
commands=python bin/test []
python bin/doctest []</code></pre></blockquote>Here we are just using the default environments (py24-py32, Jython, PyPy) and then running our test suite in each. The square brackets are important - our bin/test is programmed in such a way to accept a testname as an argument, it will then run just the specified test(s). The brackets allow us to replicate this behaviour with Tox, instead of running the whole test suite every time. However, Tox makes it very easy to define a custom environment. One such that we could use is:<br />
<blockquote><pre style="font-family: "Courier New",Courier,monospace;"><code>[testenv:py27-32-gmpy]
basepython=/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7
deps = http://gmpy.googlecode.com/files/gmpy-1.14.zip
commands = python bin/test []</code></pre></blockquote>SymPy can optionally use <a href="http://code.google.com/p/gmpy/">gmpy</a> to provide better ground types. This test environment tests it with a 32-bit version of Python 2.7; the "basepython = " allows us to specify the path to a specific python interpreter to use. The "deps" command specifies dependencies (multiple dependencies should be in multiple lines, like for "commands" above). It can download them automatically using pip, or you can specify a zip/tarball like above. You can also see the <a href="http://tox.testrun.org/en/latest/config.html">full Tox specification</a>, but these are the most important commands and the ones most likely to be used. You can also see the <a href="https://github.com/sympy/sympy/blob/master/tox.ini.sample">tox.ini.sample file we use</a>, it has some more examples. (Note: this is how configuration files should be handled in DCVS': a "canonical" .sample file, which should then be copied to .ini and modified to needs. The .ini file itself should be ignored by your DCVS)<br />
<br />
<span style="font-size: large;">Using Tox</span> <br />
<br />
Tox works by creating a virtualenv for each environment specified, by default in a .tox directory, so there is no way it can mess up your system. It will reuse the existing environments, but it's possible to force a rebuild with "tox --recreate". You normally run Tox with a simple "tox", which runs all the environments listed in "envlist". You can optionally specify particular environments with -e (so, "tox -e py25,py26" will run just py25 and 26). A neat feature here is that you don't need to have an environment in your default envlist to run it with "-e". <br />
<br />
Now obviously, the basic usage is a simple "tox". That will build all the default environments and run the whole test suite in each. Unfortunately, our test suite takes around 10 mins to run on average hardware and we currently support 3 Python versions (to go up to 4 or 5 when we finally get Python 3 support). If we add in {gmpy / no-gmpy} and even {32bit, 64bit}, we quickly get to an unrealistic amount of combinations to run. Yes, it is important to run these tests occasionally, but it isn't realistic to expect a developer to run all of them for every change they make (that will be the job of our CI server, after all). Still, it's important to test at least some basic testing for every change. As such, <b>I advocate running py25-py27 by default, every time</b>. In my opinion, this is a good compromise between time and testing coverage. If a bug occurs in some version, one can then research it further by running more variations (with gmpy, on 32bit etc) or preferably just fix it if possible. <br />
<br />
Another useful feature is that our test suite supports running just specific tests (as mentioned above). Depending on the code changes, it might be enough to run just the appropriate test suite. This will drastically reduce the time required (and will allow the developer to run in all versions, rather than the reduced list above). Alternatively, if just one test fails after a change, it's easier to debug it if you can run just the specific test. <br />
<br />
That's basically it! Tox is very simple to use and quite effective. The biggest challenge is probably persuading developers to use it, but the value is clear. It's also invaluable for testing before a release, per the words of <a href="http://vperic.blogspot.com/2011/06/gsoc-week-3.html?showComment=1307929658142#c1839748220786915622">the maintainer of SymPy</a>, Aaron.</div>Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-8814696591561172392.post-7288630260928708902011-06-11T15:54:00.002+02:002011-06-13T11:29:45.541+02:00GSoC: week 3<div dir="ltr" style="text-align: left;" trbidi="on">This week I've worked on adapting my work into commit-worthy chunks. The results are pushed to my fork on Github, you can see them <a href="https://github.com/vperic/sympy/commits/porting">here</a>. I've tried to make the commits fairly self explanatory - the first several are just fixing the warnings "-3" produces in a matter that's Python 2.5+ compatible. Unfortunately, Googling the error didn't produce much information in most cases; because of this, I've created a page where I've collected the solutions I used to solve each warning. You can find it on the <a href="http://vperic.blogspot.com/p/python-3-porting-tips-and-tricks.html">right-hand sidebar</a>. I've since also updated the version of mpmath bundled in SymPy to 0.17 (which supports Python 3 but drops support for Python 2.4, thus representing the first commit not compatible with Python 2.4).<br />
<br />
Unfortunately, here I've hit a problem. Namely, mpmath is written with a single code-base that can run both Python 2 and Python 3 unmodified (this is not the recommended course of action usually, but mpmath didn't need to change much so I guess it makes sense). Normally, this is good, but SymPy will need to use 2to3 and this creates errors in SymPy. Why? One banal example is something like:<br />
<blockquote>try:<br />
from itertools import izip<br />
except ImportError:<br />
izip = zip</blockquote>Now, when 2to3 runs on this it removes the "from itertools..." line and thus we get an error. This is just one simple case, but there are plenty of others. It's obvious that running 2to3 on compatible code is not going to produce the desired results. And this is where I'm stuck currently. 2to3 doesn't support skipping some directories, so I'll probably have to develop our "internal" script that will use lib2to3 directly. To do this, I've decided to first try to integrate automatic 2to3 into our setup.py, so that I'd have an idea on how it works. [As a side note, thanks to Lennart Regebro and Benjamin Peterson, who've helped me in <a href="http://mail.python.org/pipermail/python-porting/2011-June/000226.html">this thread</a> on the Python-porting mailing list when I asked how to speed up 2to3 -- using the latest version of it was particularly good advice as it provides ~40% speedup in my case]<br />
<br />
Once mpmath is integrated properly, I'll be able to continue on porting SymPy itself. I feel like I'm on track with the timeline in my proposal, which would be a working SymPy by the midterm evaluations.<br />
<br />
<br />
I'd also like to take a moment here to comment on the value of <b>Tox</b>. While porting yesterday, I encountered an error in a polys test. As mpmath is used heavily in this code, I thought it was something to do with the new version. Later, I realized it occurs in master too and quickly bisected it down. And this is where the trouble started, as no one could reproduce the error, with any combination of Python version, architecture, ground-types and cache used. Digging down deeper, we found that the hashes of some functions really were different for everyone, though it was just me getting the error. In the end, someone else confirmed it - Python 2.6, 64-bit, with python ground types (and only on Linux, I think).<br />
<br />
The error itself was nasty - <strike>int(1) and long(1) do not have the same hash</strike> <i>[EDIT: See Aaron's comment below for the details]</i> (I was told similar problems occurred before as well). As the underlying hash was hashing lists of lists (possibly deeply nested), the fix was to convert it to tuples of tuples and then hash it. The patch was written by asmeurer, and the whole discussion can be found on the <a href="http://code.google.com/p/sympy/issues/detail?id=2472">issue page</a> (and the IRC logs <a href="http://kottnet.net/irclogs/freenode/chn-%23sympy/2011-06-10.log">from yesterday</a>). The moral of the story? Use Tox! How else would we find something that only appears in such specific circumstances. </div>Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-8814696591561172392.post-20933281593546746522011-06-04T19:32:00.001+02:002011-06-04T19:48:51.097+02:00GSoC: week 2<div dir="ltr" style="text-align: left;" trbidi="on">This has been a particularly uninteresting week of work, as I've expected. I have done some work towards making SymPy code Python 3 compatible, but there are no visible results - it just doesn't run in Python 3. As soon as it's possible to at least run SymPy in Python 3, I will begin publishing my branch. A hurdle is that SymPy should still be supporting Python 2.4 until the release of 0.7.0. Unfortunately, this release has been "imminent" for a while now. Initially, I was hoping to start integrating my work in small chunks as soon as possible. However, as this is the second week already, I've had to start some work in parallel. I only hope merging later will not be a problem.<br />
<br />
It should also be noted that constantly re-running 2to3 is not a pleasant task. While this has some implications for testing, it's more importantly slowing down my work.</div>Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-8814696591561172392.post-20899951374440543192011-05-28T18:48:00.002+02:002011-06-11T15:56:41.612+02:00Continuous integration and SymPy: Buildbot or Jenkins (with Tox)<div dir="ltr" style="text-align: left;" trbidi="on"><blockquote><i>In software engineering, <b>continuous integration</b> (<b>CI</b>) implements </i><i>continuous processes of applying quality control — small pieces of effort, applied frequently. Continuous integration aims to improve the quality of software, and to reduce the time taken to deliver it, by replacing the traditional practice of applying quality control </i><i>after completing all development. <b>--</b></i><b>Wikipedia, <a href="http://en.wikipedia.org/wiki/Continuous_integration">Continuous integration</a></b></blockquote>The above definition might seem a bit strange and archaic - isn't most open-source software developed in small chunks and merged in as soon as possible? That's why we have git and other distributed version control systems, as they make this work much more manageable. Continuous integration also means testing, which is something that is especially important for a library, and that's where we hit the first snag. SymPy has a policy that all tests must pass before making a change, which is a valid policy, but the simple fact of life is that SymPy is supposed to work on various platforms across multiple Python versions (and different ground types!). This means that a single developer cannot reasonably check all possible combinations and in the long run introduces subtle bugs in the code, especially in eg. older Python versions. This is where a continuous integration server comes in.<br />
<br />
<div style="text-align: left;">The goal of a continuous integration server is to, well, continually integrate. It controls some slaves, gives them tasks and collates the results. Usually, this means building the project and running the test suite, but it can be anything. This process is then repeated nightly, or after every commit, or started manually (eg. with specific parameters); in general, CI servers are very powerful and extendable, to fit the needs of specific projects. Part of my GSoC project was to investigate which CI server could be used with SymPy. I have considered <a href="http://trac.buildbot.net/">buildbot</a>, which was used in SymPy previously, and <a href="http://jenkins-ci.org/">Jenkins</a>. </div><div style="text-align: left;"><br />
</div><div style="text-align: left;">Current SymPy workflow is for another developer to run the test suite on a given pull request before (thinking of) merging. To automate this process, a helper tool called <a href="https://github.com/sympy/sympy-bot">SymPy-bot</a> has been developed. It is a simple Python script which can list all the pull requests and test a particular one. The results are presented in a table on pastehtml and a comment is automatically made in the pull request. In essence, a poor man's continuous integration. :)</div><br />
<div style="text-align: left;"><span style="font-size: large;">Buildbot</span></div><div style="text-align: left;"><br />
</div><div style="text-align: left;"><span style="font-size: small;">Buildbot was used </span>by SymPy before, so it was a natural first choice (the fact that it's widely used, notably by Mozilla and Chrome, is another big plus). Buildbot has a classic master/slave structure, which means that each slave has to be setup separately, with the appropriate environment prepared in advance. As a test, I have created a local buildbot and some slaves and played around with them. Buildbot presents the information in a table format by default (<a href="http://build.chromium.org/p/chromium/waterfall">like this</a>), but can easily send it by mail or to an IRC channel or any combination of the above. It is in general a very robust project. A big advantage is the existence of a "TryScheduler", which applies a given patch and runs tests. This closely approximates the current SymPy workflow and I consider it a very desireable feature. </div><div style="text-align: left;"><br />
</div><div style="text-align: left;">Unfortunately, the robustness of Buildbot comes at the cost of complexity. Setting up a build slave is a non-trivial task and in the end I decided to take a look at Jenkins first, before continuing to work with Buildbot.</div><div style="text-align: left;"><br />
</div><div style="text-align: left;"><span style="font-size: large;">Jenkins</span></div><div style="text-align: left;"><br />
</div><div style="text-align: left;">Jenkins (formerly Hudson) is a CI server written in Java, providing much of the same functionality as Buildbot. Unfortunately, it does not support Python natively. This is where <a href="http://tox.testrun.org/en/0.9/index.html#">Tox</a> comes in. Tox is an "automation project" for Python programs, which uses virtualenv to create different environments where the program can be tested in. It is extremely easy to setup, the following tox.ini file is all I needed to have (and appropriate Python versions installed, of course): </div><div style="text-align: left;"></div><blockquote><div style="text-align: left;">[tox]<br />
envlist = py25, py27, docs<br />
[testenv]<br />
changedir=bin<br />
commands=python test []<br />
[testenv:docs]<br />
commands=python doctest []</div></blockquote>It is then run with "tox", which automatically creates the necessary virtualenv's (reusing them if they already exist, of course) and then executes the given commands. The [] brackets allow us to replicate the behaviour of "./bin/test hydrogen" (runs just hydrogen tests) with "tox hydrogen". I like Tox a lot, as it makes it easy for a single developer to test many Python versions (as long as they are installed). Alternatively, it is easy to modify the tox.ini file to test with just the available Python interpreters.<br />
<br />
Moving on, Tox also provides seamless integration with Jenkins (through the use of a "multi-configuration project") which then provides all the features expected from a CI server, including automatic builds, nice presenting of data and so on (speaking of presentation, it would be a good idea to have our test tool support JUnitXML, which shouldn't be too hard). One disadvantage of the Tox/Jenkins combination is the lack of a TryScheduler like the one Buildbot has. I have spoken with the developers, and they will add it to their Github plugin (eventually). In the mean time, it should be possible to manually program the functionality we need (it is possible to request a build with a parameter, which could be the name of a pull request) accessing Github directly. As this is something SymPy-bot does, I wanted to speak with Ondrej how to do this exactly.<br />
<br />
In conclusion, Tox/Jenkins impressed me enough not to go back to Buildbot. Tox is very simple to configure and can be run locally. Jenkins appears simpler than Buildbot to setup and maintain, while offering almost the same functionality. The lack of a try scheduler is unfortunate, but as we've basically already engineered a solution to the same problem, I'm confident a solution will be found. In general, though, I feel that using Tox/Jenkins is an excellent choice for any Python project concerned with compatibility. <i>[Update: I've written a short guide to using Tox with SymPy on the Wiki, you can read it <a href="https://github.com/sympy/sympy/wiki/Using-Tox">here</a></i>]</div>Unknownnoreply@blogger.com2tag:blogger.com,1999:blog-8814696591561172392.post-69025571101144101572011-05-10T11:01:00.001+02:002011-05-10T11:02:54.062+02:00./setup.py GSoC/blog<div dir="ltr" style="text-align: left;" trbidi="on">[title shamelessly stolen from my mentor, Ronan Lamy]<br />
<br />
As hinted to by the title, I've been accepted for the <a href="http://www.google-melange.com/gsoc/homepage/google/gsoc2011">Google Summer of Code 2011</a>. My work will be on <a href="http://sympy.org/">SymPy</a>, which is a Python library for symbolic mathematics (think Maple or Mathematica), and my goal is to convert it to Python 3. Namely, I will try to support all Python versions starting from 2.5, and this should be accomplished with the use of a single code-base (and automatic running of the 2to3 tool in the Python 3 use case). You can read my full application <a href="https://github.com/sympy/sympy/wiki/GSoC-2011-Application%3A-Vladimir-Peri%C4%87%3A-Porting-to-Python-3">here</a>.<br />
<br />
As this is a complex issue, the first order of business would be to establish an automated testing system that would be able to spot any regressions (currently, SymPy supports Python 2.4 - 2.7, and there are already various small bugs and differences in behavior based on Python version); the current contenders are <a href="http://jenkins-ci.org/">Jenkins</a> (via <a href="http://codespeak.net/tox/example/hudson.html">Tox</a>) and <a href="http://trac.buildbot.net/">Buildbot</a>. I will write-up a more detailed comparison, as related to the specific use-case of SymPy, when I've tried both. Of course, any comments, suggestions or new ideas are very welcome.<br />
<br />
My second major goal for the community bonding period (ending on the 23rd) is to work on the warnings running with "Python -3" produces. This is an important first step to Python 3 porting, as it will solve some real issues and help me understand the code better. If possible, I will also try to improve code coverage of the test suite (or, alternatively, bother the developers responsible for that piece of code), which is also important for preventing subtle bugs from appearing.<br />
<br />
Here's to running SymPy on Python 3 (<i>and</i> PyPy!) by the end of the summer!</div>Unknownnoreply@blogger.com1