Jekyll2017-11-12T14:37:29-08:00https://alex-konovalov.github.io/Dr. Alexander KonovalovSenior Research Fellow in Computational AlgebraDr. Alexander KonovalovThere is no royal way in computational algebra2017-07-23T00:00:00-07:002017-07-23T00:00:00-07:00https://alex-konovalov.github.io/posts/2017/07/23/no-royal-way<p>This post is based on my answer to the question
<a href="https://math.stackexchange.com/q/2353307/">“Royal way to learn algorithmic/computational/computer algebra”</a>
asked at Mathematics Q&A site by Jakob Werner. The questions asks for a book
recommendation, and then more specifically asks whether is it important to
have a book focusing on one CAS explicitly; which CAS to decide for;
where does the programming experience come from; and how to check the
written code is good.</p>
<p>The background described in the question is not uncommon - with
the growth of computational disciplines many researchers have to
learn programming on the job, without any formal training during
their degree studies. Below I will try to give some advice applicable
to computational algebra.</p>
<p>In principle, one could be a researcher in algorithms without doing any
actual programming. For example, one could study complexity of
algorithms and their asymptotic behaviour which will be beyond any
feasible computer implementations. On the other hand, programming skill
would be an advantage. It may help in understanding how to present
algorithms for their practical implementations, or to find worst case
performance examples; it may allow to play with things to check facts
and better understand the theory, and to find examples and
counterexamples.</p>
<p>Now on the specific questions.</p>
<h1 id="is-it-important-to-have-a-book-focusing-on-one-cas-explicitly">Is it important to have a book focusing on one CAS explicitly?</h1>
<p>No, it’s not. A book dedicated to a particular CAS may not even exist in
a relevant area. Books are certainly important, but not only they.
Theoretical descriptions in the textbook could be quite high-level and
not hinting at subtle aspects of their practical implementation. Code
provided with textbooks may not run in the next version of the software.
It is also important to look at manuals, teaching materials, comments in
the code, or the code itself (provided the system is open source).</p>
<h1 id="which-cas-to-decide-for">Which CAS to decide for?</h1>
<p>It depends on your area of research, on problem(s) which you’d like to
work on, and on availability on the relevant functionality in
appropriate CAS. An important factor is being open source. Besides CAS,
I recommend to learn at least one general purpose programming language,
for example Python. It has a large community and a plenty of teaching
materials available. If you may need to learn several CAS, it’s better
to start to learn with the one which has a less steep learning curve,
seems more usable, has active community, accessible teaching materials,
etc. In no particular order, you may want to look at
<a href="https://www.gap-system.org/">GAP</a>,
<a href="http://www.math.uiuc.edu/Macaulay2/">Macaulay2</a>,
<a href="http://www.sagemath.org/">SageMath</a>,
<a href="https://www.singular.uni-kl.de/">Singular</a>,
and others.</p>
<h1 id="where-does-the-programming-experience-come-from">Where does the programming experience come from?</h1>
<p>Reading existing code, preferably short and documented functions, is
useful, but it’s not enough. As Brian Kernighan and Dennis Ritchie said
in the first edition of The C Programming Language in 1978,
<a href="https://www.artsy.net/article/ruse-laboratories-brian-kernighan"><em>“The only way to learn a new programming language is by writing programs in it”</em></a>.
You need to actually practice in writing your own code. Take some
algorithm and try to implement it from scratch. Check that the result
matches theory and/or existing implementation. Indeed, why not to try to
implement the Buchberger algorithm in Singular, or the orbit enumeration
algorithm in GAP? You will not have to re-implement everything since the
underlying computer algebra system will provide you with basic objects,
so you will start with some higher level, and will learn a lot about its
functionality in the process. Even if it will be not performing so well
as the native one, that would be a useful exercise.</p>
<h1 id="how-to-check-if-i-have-written-good-code">How to check if I have written good code?</h1>
<p>I would reformulate it as “How to learn to write a good code” at try to
use some good practices from the beginning. You can find some useful
tips looking for reports from various “What makes good code good”
discussions on the Software Sustainability Institute website
<a href="https://www.software.ac.uk/search/node?keys=what+makes+good+code+
good">here</a> (for example, see
<a href="https://www.software.ac.uk/blog/2016-09-28-what-makes-good-code-
good-emcsr-2014">here</a> and
<a href="https://www.software.ac.uk/blog/2016-10-07-what-makes-good-code-
good-mozfest">there</a>). I recommend to attend some <a href="https://software-carpentry.org/workshops/">Software Carpentry
workshop</a> to learn more about
these practices. A typical workshop for beginners does not require prior
knowledge of the tools used, lasts about two days, and covers UNIX
shell, version control and programming with one of the languages (see
the list of lessons <a href="https://software-carpentry.org/lessons/">here</a>). It
is “programming <strong>with</strong> language”, not “programming <strong>in</strong> language”,
since the emphasis is on general programming practices, being taught
with the help of a particular language. You can check for the future
workshops in your region
<a href="https://software-carpentry.org/workshops/">here</a>.</p>
<p>In addition, it’s useful to follow mailing lists and Q&A sites for CAS
of your interest, get in touch with the community of its users and
developers (for example, at events like <a href="http://gapdays.de/">GAP days</a>
or <a href="https://wiki.sagemath.org/Workshops#Sage_Days">SageMath days</a> or at
training events like such <a href="https://www.codima.ac.uk/schools/">training schools</a>).
If the system has an issue tracker, perhaps some issues are labeled as
newcomer-friendly (like we do in GAP
<a href="https://github.com/gap-system/gap/issues?q=is%3Aissue+is%3Aopen+label%3Anewcomer-friendly">here</a>) so you may try to work on them and then get
some feedback from developers.</p>Dr. Alexander KonovalovThis post is based on my answer to the question “Royal way to learn algorithmic/computational/computer algebra” asked at Mathematics Q&A site by Jakob Werner. The questions asks for a book recommendation, and then more specifically asks whether is it important to have a book focusing on one CAS explicitly; which CAS to decide for; where does the programming experience come from; and how to check the written code is good.GAP Tutorial in Birmingham2017-06-10T00:00:00-07:002017-06-10T00:00:00-07:00https://alex-konovalov.github.io/posts/2017/06/10/gap-tutorial<p>Jointly with Sergey Shpectorov (Birmingham), I am organising a
<strong><a href="https://www.codima.ac.uk/gsta2017/">GAP Tutorial in Birmingham</a></strong>
on August 13th-14th. It will immediately follow
<a href="http://www.groupsstandrews.org/2017/">Groups St Andrews 2017 in Birmingham</a>,
and I hope that some participants will be interested to stay for it.
Participation is open to everyone, and there is a limited financial support
available to PhD students from UK Universities, provided by
<a href="https://www.codima.ac.uk/">CCP CoDiMa</a>. Please see
<em><a href="https://www.codima.ac.uk/gsta2017/">this page</a></em> for further details
and registration link.</p>Dr. Alexander KonovalovJointly with Sergey Shpectorov (Birmingham), I am organising a GAP Tutorial in Birmingham on August 13th-14th. It will immediately follow Groups St Andrews 2017 in Birmingham, and I hope that some participants will be interested to stay for it. Participation is open to everyone, and there is a limited financial support available to PhD students from UK Universities, provided by CCP CoDiMa. Please see this page for further details and registration link.Data Carpentry lesson on spreadsheets2017-05-04T00:00:00-07:002017-05-04T00:00:00-07:00https://alex-konovalov.github.io/posts/2017/05/04/dc-spreadsheets<p><a href="http://www.datacarpentry.org/">Data Carpentry</a> just published their lesson
<a href="http://www.datacarpentry.org/spreadsheet-ecology-lesson/">“Data Organization in Spreadsheets”</a>
via Zenodo (see <a href="http://doi.org/10.5281/zenodo.570047">here</a>). I’ve made a small
contribution to that lesson as a part of my Data Carpentry instructor training
checkout last year. Amazing to see how easy it is now to add it to
<a href="http://orcid.org/0000-0001-5299-3292">my ORCID record</a> using DataCite search.</p>Dr. Alexander KonovalovData Carpentry just published their lesson “Data Organization in Spreadsheets” via Zenodo (see here). I’ve made a small contribution to that lesson as a part of my Data Carpentry instructor training checkout last year. Amazing to see how easy it is now to add it to my ORCID record using DataCite search.Software Carpentry: back by popular demand2017-05-04T00:00:00-07:002017-05-04T00:00:00-07:00https://alex-konovalov.github.io/posts/2017/05/04/swc-is-back<p>Jointly with <a href="https://www.st-andrews.ac.uk/capod/">CAPOD</a>,
<a href="https://www.st-andrews.ac.uk/libraryblog/research-computing/">Research Computing</a>,
and <a href="http://www.hutton.ac.uk/staff/leighton-pritchard">Leighton Pritchard (The James Hutton Institute)</a>,
we have recently run a
<a href="https://alex-konovalov.github.io/2017-03-23-standrews/">Software Carpentry workshop</a>
where we taught Unix shell, version control with Git, and programming
with Python to more than 20 staff members and postgraduate students of the
University of St Andrews. The workshop was very quickly oversubscribed, and
to meet these demands, we will run another one on May 18th-19th.
Please see <a href="https://alex-konovalov.github.io/2017-05-18-standrews/">this page</a>
for further details and registration instructions.</p>
<p>Please come if you are interested in any of the mentioned topics.
There are only several places left, so I advise to register as soon
as possible! You do not need to have any prior knowledge of any of
the topics. Also, let me emphasise that you can use version control
in multiple ways which are not related to programming, to example,
for collaborative writing of papers and grant applications in LaTeX,
or for maintaining a website like this one.</p>Dr. Alexander KonovalovJointly with CAPOD, Research Computing, and Leighton Pritchard (The James Hutton Institute), we have recently run a Software Carpentry workshop where we taught Unix shell, version control with Git, and programming with Python to more than 20 staff members and postgraduate students of the University of St Andrews. The workshop was very quickly oversubscribed, and to meet these demands, we will run another one on May 18th-19th. Please see this page for further details and registration instructions.Teaching good mathematical software practices at CoDiMa training school2016-12-15T00:00:00-08:002016-12-15T00:00:00-08:00https://alex-konovalov.github.io/posts/2016/12/15/codima2016-report<p>On October 17-21, 2016 we organised the
<a href="http://www.codima.ac.uk/school2016/">Second CoDiMa training school in Discrete Computational Mathematics</a>
in Edinburgh (our first school took place in Manchester in November 2015).
This time it was hosted at the
<a href="http://www.icms.org.uk/">International Centre for Mathematical Sciences</a>,
and had been attended by 26 learners representing 11 institutions around
the UK. The majority of them were PhD students in mathematics and computer
science. Their participation was supported by the CoDiMa project, which
covered their travel, accommodation and subsistence needed to attend
the school.</p>
<p>The school started with the hands-on
<a href="https://widdowquinn.github.io/2016-10-17-edinburgh/">Software Carpentry workshop</a>
covering the UNIX command line and version control (instructed by
<a href="http://tarut.in/">Alexey Tarutin</a> and <a href="http://www.hutton.ac.uk/staff/leighton-pritchard">Leighton Pritchard</a>),
and the <a href="http://alex-konovalov.github.io/gap-lesson/">Software Carpentry lesson on GAP</a>
given by myself (for published versions of these lessons, see
<a href="http://doi.org/10.5281/zenodo.57544">here</a>,
<a href="http://doi.org/10.5281/zenodo.57467">here</a> and
<a href="http://doi.org/10.5281/zenodo.167362">there</a> on Zenodo).
The 2-days Software Carpentry workshop created the basis for more in-depth
explanation of the following topics on GAP and computational algebra during
the remaining part of the week:</p>
<ul>
<li>debugging and profiling</li>
<li>advanced GAP programming</li>
<li>GAP type system</li>
<li>distributed parallel calculations</li>
<li>demonstration of the new GAP Jupyter interface</li>
<li>examples of some algorithms and their implementations</li>
</ul>
<p>delivered by <a href="https://caj.host.cs.st-andrews.ac.uk/">Christopher Jefferson</a>,
<a href="https://www.cs.st-andrews.ac.uk/directory/person?id=sal">Steve Linton</a>,
<a href="https://www.morphism.de/~markusp/">Markus Pfeiffer</a>,
<a href="http://www-circa.mcs.st-and.ac.uk/~waw7/">Wilf Wilson</a>
and myself.</p>
<p>Furthermore, <a href="http://homepages.warwick.ac.uk/staff/J.E.Cremona/">John Cremona</a>
gave a presentation of LMFDB (The L-functions and modular forms database
project) which offered interesting insight into the internals of designing
and maintaining mathematical database such as LMFDB. Also on Thursday
<a href="https://www.lri.fr/~pons/en/">Viviane Pons</a> (Université Paris-Sud) gave
an introduction to SageMath, and on the final day we had
<a href="http://mikecroucher.github.io/MLPM_talk/">“Is your research software correct?”</a>
talk by the EPSRC Research Software Engineering Fellow
<a href="http://www.walkingrandomly.com/">Mike Croucher</a> (Sheffield), and a panel
discussion joined by
<a href="https://www.software.ac.uk/about/people/neil-chue-hong">Neil Chue Hong</a>,
who is the director of the
<a href="https://www.software.ac.uk/">Software Sustainability Institute</a>.</p>
<p>For further details, please see the school’s
<a href="http://www.codima.ac.uk/school2016/">webpage</a> which contains links to the
presentations and supplementary materials for all school’s programme.
You can also find
<a href="https://storify.com/CIRCA_StAndrews/codima-2016">all #codima2016 tweets on Storify</a>
(they provide very good day by day coverage of the whole week), and school
photos courtesy of ICMS
<a href="https://www.flickr.com/photos/icmsnews/sets/72157674045310781">here on Flickr</a>.
It was a really exciting event, and it is a great pleasure to thank everyone
involved: all participants, speakers, instructors, helpers;
Software Sustainability Institute and personally Giacomo Peru
who coordinates Software Carpentry activities in the UK;
all contributors to Software Carpentry lessons that we taught;
ICMS staff for welcoming us and being attentive to all our requests;
Blue Sky Catering for nice lunches;
and <a href="http://www.vittoriagroup.co.uk/vittoriabridge/">Vittoria on the Bridge</a>
for the school dinner!</p>
<p>We plan at least three more such schools in 2017-2019.
Besides training events, we also organise annual workshops.
The first one was in the form of
<a href="http://gapdays.de/gap-sage-days2016/">GAP-SageMath days</a>
at St Andrews in January 2016, and the next one is
<a href="http://opendreamkit.org/meetings/2017-01-16-ICMS/">“Computational Mathematics with Jupyter”</a>
which we organise jointly with the Horizon 2020 project
<a href="http://opendreamkit.org/">OpenDreamKit</a> at the
<a href="http://www.icms.org.uk/">International Centre for Mathematical Sciences</a>
in Edinburgh on January 17-20, 2017.</p>Dr. Alexander KonovalovOn October 17-21, 2016 we organised the Second CoDiMa training school in Discrete Computational Mathematics in Edinburgh (our first school took place in Manchester in November 2015). This time it was hosted at the International Centre for Mathematical Sciences, and had been attended by 26 learners representing 11 institutions around the UK. The majority of them were PhD students in mathematics and computer science. Their participation was supported by the CoDiMa project, which covered their travel, accommodation and subsistence needed to attend the school.Publishing Software Carpentry lesson on GAP2016-11-22T00:00:00-08:002016-11-22T00:00:00-08:00https://alex-konovalov.github.io/posts/2016/11/22/publishing-software-carpentry-lesson-on-gap<p>Following the <a href="http://www.codima.ac.uk/school2016/">2nd CoDiMa training school</a>,
I have published the
<a href="http://alex-konovalov.github.io/gap-lesson/">Software Carpentry lesson on GAP</a>
via Zenodo: see <a href="http://doi.org/10.5281/zenodo.167362">10.5281/zenodo.167362</a>.
The lesson is based on the problem of determining an average order of an element
of a finite group, and finding examples of groups for which the average order of
their elements is an integer. First I have heard about this problem when Steve
Linton used it in a talk in order to quickly demonstrate some GAP features to a
general scientific audience. I have tried to expand on it in my talk in Newcastle
in May 2015 (see the blog post
<a href="http://www.codima.ac.uk/2015/07/01/average-order-of-group-elements-a-demo-of-test-driven-development-in-gap/">here</a>),
and decided to proceed with it.</p>
<p>Indeed, the problem of determining an average order of the element of the group
is simple enough to not to distract learners too much from the intended learning
outcomes of the lesson. An undergraduate algebra course is sufficient for its
understanding. Moreover, those not familiar with the group theory still should
be able to follow the lesson just by grasping the idea that there is a mathematical
structure called group, and we need to find an average value of a certain numerical
parameter associated with each element of it. On the other hand, those with
sufficient theoretical background will hopefully enjoy seeing how the initial
naive implementation is being refined several times during the lesson, and how
theoretical insights are giving much more significant advances than minor code
optimisations or just getting more cores.</p>
<p>The lesson starts with formulating the problem of finding examples of groups
such that the average order of their element is an integer. It first explains
how to work with the GAP command prompt, demonstrates some basic language
constructions and explains how to find necessary information in the GAP help
system. At this point using the command line we establish a rough prototype of
the code to compute an average order of a group, and tried several examples,
none of them yielding an integer.</p>
<p>Next, it discusses that the command line usage is good only for rapid prototyping,
and explains how to create GAP functions, place them into a file, and read that
file into GAP. After that, the initial implementation is used to create a
regression test: GAP runs it by comparing the actual output with the reference
output, and will fail the test in case of any discrepancies. Testing the code
is a topic that usually escapes beginners’ attention, and I am really excited
about managing to cover it as a part of the introductory GAP lesson. I explain
the “make it right, than make it fast” paradigm, and show how to create and run
regression tests in GAP after the first naive implementation is available. To
demonstrate test failures, I deliberately mixed up function names to break the
test, and it was a real pleasure when the audience pointed that out before I
even managed to re-run the test and demonstrate that it failed.</p>
<p>Having the improved and tested implementation, we start systematic search for
finite groups with an integer average order of an element using the
<a href="http://www.gap-system.org/Packages/sgl.html">GAP Small Groups Library</a> which
contains, among others, all 423 164 062 groups of order at most 2000 except 1024.
At the same time, the lesson introduces modular programming and shows how one can
design a system of functions to perform the search in a way that one could re-use
most of the code and only develop a new function to test a single group to deal
with another search problem. Then the first interesting example (a group of order
105 such that the average order of its elements is 17) is discovered! The next
obstacle is to check all 56092 groups of order 256, however, a short theoretical
observation shows that we can exclude groups of prime power order from the search,
as they will never have an integer average order of an element. After modifying
the code to skip such orders, the search continues, and then another example
(a group of order 357) is found. Discovering another known group with this property
is left as one of the exercises.</p>
<p>The lesson finishes with explaining how the knowledge about the object can be
stored in it. For example, once the average order of an element of a group is
calculated and stored in the group, it can be next time retrieved at zero cost,
avoiding redundant calculations.</p>
<p>Of course, it is not possible to cover everything in a several hours long course,
but it fits really well into the week-long CoDiMa training school like
<a href="http://www.codima.ac.uk/school2016/">this</a>. It prepares the audience to hear
about more advanced topics during the rest of the week: debugging and profiling;
advanced GAP programming; GAP type system; distributed parallel calculations;
examples of some algorithms and their implementations, etc. Also, staying for
the whole week of the school, everyone has plenty of opportunities to ask
further questions to instructors.</p>
<p>What next? So far I am only aware that it has been taught twice (by myself) at
two annual
<a href="http://www.codima.ac.uk/schools/">CoDiMa training schools in computational discrete mathematics</a>.
I can surely teach it myself, but is it written clearly enough to be taught by others?
Is it possible for the reader to follow it for self-studying? Is there any
introductory material missing, or is there an interest in having more advanced
lesson(s) on some other aspects of the GAP system? If you would like to
contribute to its further development, issues and pull requests to its repository
on <a href="https://github.com/alex-konovalov/gap-lesson">GitHub</a> are most welcome! Also,
we invite collaborators interested in developing a lesson on
<a href="http://www.sagemath.org/">SageMath</a>: please look at
<a href="https://github.com/alex-konovalov/sage-lesson">this repository</a> and add a
comment to <a href="https://github.com/alex-konovalov/sage-lesson/issues/1">this issue</a>
if you’re interested in contributing.</p>Dr. Alexander KonovalovFollowing the 2nd CoDiMa training school, I have published the Software Carpentry lesson on GAP via Zenodo: see 10.5281/zenodo.167362. The lesson is based on the problem of determining an average order of an element of a finite group, and finding examples of groups for which the average order of their elements is an integer. First I have heard about this problem when Steve Linton used it in a talk in order to quickly demonstrate some GAP features to a general scientific audience. I have tried to expand on it in my talk in Newcastle in May 2015 (see the blog post here), and decided to proceed with it.