By Ole Moeller-Nilsson, PhD Astrophysics, S2DS Lecturer and Software Analyst at AON Benfield

I have worked on larger software projects both for the scientific community as well as for the corporate world. Recently a friend of mine asked me what I thought the differences, in terms of the challenges as a software developer are. It was not totally straightforward for me to answer that question right away and in the end I decided it was maybe worth writing my thoughts on this topic down. I am sure there are many more who have experiences in both areas and if you are one of these, I am also really curious to hear what your answer would be?

First of all, I think there are two broad areas that one can talk about here: the more technical challenges, or challenges of understanding, and the interpersonal or communication challenges that arise from the working culture. There are sufficient differences in both of these areas between the science world and business world to make each one relevant.

The importance of domain expertise

The one challenge that both have in common is that of understanding the domain. How important is it that a software developer understands or even is an expert in the domain that the software is written for? This is actually a quite old question and there have been several answers formulated, most notable that of Eric Evans who realised the importance of a common understanding between domain experts and software developers, which gave rise to the concept of the “Ubiquitous Language” and domain driven design.

When I began working as a developer for the life insurance industry I felt a little lukewarm about the actual industry. Sure, I love mathematics, but insurance? I was very sceptical I would really find it exciting enough to really get into it. Already during the interview my boss told me that I should not worry about that too much – after all I was going to be a developer and not an actuary. This reassured me but in retrospect I think this was a mistake. A good software developer, definitely if you want to be someone who makes a difference, has to be very interested in the domain, eager to learn it (and actually do so). This is not just because at the end of the day finding the domain interesting can be a great motivator in periods when the software tasks have become a bit repetitive and dull. Mainly it allows you to feel confident about the software you are working on, be faster in grasping the requirements and making useful comments and suggestions for improvements. You will be much more likely to really make a difference and if you have something to say domain experts are more likely to listen and understand you. So, this is something that is an important point for both: software development in science and industry. From my experience I would say that it is even more important for science than for business.

The lack of appreciation of software craftsmanship in academia

One area where I strongly feel working on a software project for science has unique challenges is that of understanding of software and the process of writing software. I feel that there is very little understanding of that in science, whereas there is significantly more of that in business. While a manager will often be too optimistic about timescales and budgets involved in a software project, she will still more or less have a broad idea of what is involved and that it is not just a matter of hacking out a few lines of code. The business world has been too infused with large scale software projects for people to be ignorant of the main issues. However, in science, coding is rarely seen as more than a means towards an end (a scientific paper). I would risk a guess and say that significantly more than 90 per cent of all code written for science is written without any intent of reuse. Even if there is such an intent there is rarely an appreciation about what is involved to create quality software. Of course there are large scale science projects (especially in Particle Physics) that need large scale software projects and these are often done to a high standard of quality. But an understanding of quality software development is not inherent in the scientific work culture, and only exists due to a few individuals who are often much more programmers than scientists. And so, working as a software developer for science, this is one of the challenges: to raise awareness that software is an engineering product that should be finished to a high quality standard. It requires the scientific domain experts (the scientists) to appreciate that methodologies that have been developed in the business world need to be imported into the project and that all science experts involved in the project have to be open to learn about those and in many cases also adjust their own mode of working. Doing that is a tremendous challenge.

In the more technical domain, I think the biggest challenge lies in finding this common ground of understanding, to define this common language that everyone can understand and that is both close to the software domain and the expert domain. In my experience it is often the case that it starts with an appreciation that neither areas are trivial or easy – and that some effort in learning is needed. I have met many who thought that writing software is easy. Make no mistake: it is not. A good place to start may be to organise seminars for information exchange, workshop and working pairs (similar to pair programming – a technique used by many software companies to both ensure quality of work and improve dissemination of knowledge).

In terms of the technological side, the most challenging aspects that are usually underestimated are that of software design and architecture, code quality, databases and their use beyond the most basic. These are vast topics and one is never finished learning about them.

Does corporate culture stifle creativity?

One the side of communication and culture I found that it can be very difficult to get used to a more restricted and totally goal focused culture, as is common in the business world. Here it seems to me as if the scientific work culture has a lot to offer. Many corporations specifically suffer from a work culture that is likely to stifle innovation and creativity. And also, I doubt that an extremely goal focused environment actually really improves productivity. The business world can learn a lot here from science (see article in the Economist by Schumpeter). Given the playful environment that many tech startups offer their employees it is clear that these have understood this to a large degree. Conversely it is sometimes hard for scientists to understand that a piece of work is only valuable if it is done fast rather than done thoroughly. Given what I wrote about quality of software the question is if this is also true for software development? I think largely it is. While being thorough is important it is for example easy to waste time on performance optimisations that are actually not needed.

In summary I think the biggest difference and challenges lie in the understanding of the software/technical domain and the adjustment to the difference in work culture. At times these challenges can be a little frustrating – especially if there is a lack of common understanding of the main issues – but they can also be extremely rewarding. Successfully bridging the gap as it were and working in an environment where people have very different backgrounds, weaknesses and strength and yet have a fluent common understanding and great communication can give you a great buzz.