It is possible to take many different project management courses online or offline, but all of them, when it comes to the topic of research and development will only glance at or ignore it completely. This is how the PM course would like you to imagine a project:

This is how a small part of an R&D project actually looks like:

How to even start managing something like this? Here are some guidelines that helped me personally over the years, as well as some general advice.

Clear cut-off points

A few years ago, I was working on a high precision measurement project that seemed like it had dozens of potential solutions, but none of them led to any progress. My boss at a time told me one simple sentence that had a significant impact: “Ignore all problems that have an error smaller than 10 microns for the time being”.

That advice helped me focus on the 20% of tasks that were relevant and ignore the minor details at that moment. Everyone knows that 20% of work delivers 80% of results, but to define them during the R&D phase is a tiresome chore.

Don’t get emotionally involved

This is probably the most challenging point to adhere to. We all like to believe that we are rational individuals who make their decisions based on facts and meaningful arguments. Still, there are whole fields of science arising in recent decades that prove that we are not i.e., behavioral economics. It is tough to give up a beautiful idea that you’ve spent weeks or maybe even months perfecting and written 10.000 lines of code, but if it is proven to be wrong, you have to give it up, and you know it.

The question is how to deal with the death of something you’ve created. In my experience, you should write. If the idea is proven wrong, you should at least write down what you’ve learned so you can apply it in the next situation or to remind yourself not to go that path again. Worst case scenario you’ll be able to justify your time with an excellent report to your superiors or clients.

Write an R&D journal 

Every day you have to test a large number of different hypotheses, which may only slightly differ from each other. This will lead to confusion unless the critical points of the day are written down.

I will assume that if you’re reading this piece, that you’ve read plenty of professional articles where you’ve been told to write a journal, and probably thought of it as childish. I know I have, and I’ve entirely missed the point. 

If you’re working in R&D, there is a huge chance, you’re a STEM major, who rarely needed to write something coherent while studying and dismissed writing as something reserved only for liberal art majors. Here is how it’s useful to science. Writing is slow and hard, it requires to focus on one subject, and to me, it is the complete opposite of my thinking; quick and scattered. Taking ten minutes to write something at the end of the business day is a form of meditation.

It is an excellent method to sort out your thoughts and to remind yourself of bad ideas and why they didn’t work out. I like to take ten minutes before leaving work to write the key points that have been tested today and why they did or didn’t work. This gives me time to pause and ask myself what I am going to work on tomorrow.

Since I’ve started a project journal, I’ve consulted it on many occasions to see what I did in the past. I find it much more practical than scouring through my emails and reports.

Build benchmark tests and simulations

Create a relatively large test sample with fixed parameters on which new ideas could be regularly tested to follow your progress over time. If possible, automatize everything, and save in a way that will enable comparison later.

This step might seem obvious, but at this very moment, I realized that it took me six months to build a quality benchmark test on one of my recent projects.

Automate your reports

A few things in life demotivates as quickly as doing the same thing for the hundredth time. I’m positive that you’re familiar with that feeling. You’ve done the hard work, and now you need to display your results and create a report. You’re tired, and you don’t want to write another procedure to do it automatically. Instead, you open the report and edit the missing commas or extra columns manually.

It is hard to tell when it’s time to automate something, and when are you just wasting your time writing a routine that may only be used once. That’s why I use a rule of thumb if I did it more than twice manually; I’m going to spend some time in automatizing it because there is a high chance that it will come up again. The reason for that could be found in Lindy effect best explained in Nicholas Taleb’s book Antifragile:

If a book has been in print for forty years, I can expect it to be in print for another forty years. But, and that is the main difference, if it survives another decade, then it will be expected to be in print another fifty years. This, simply, as a rule, tells you why things that have been around for a long time are not “aging” like persons, but “aging” in reverse. Every year that passes without extinction doubles the additional life expectancy. This is an indicator of some robustness. The robustness of an item is proportional to its life!

Using the same logic, we could say that if the procedure was done manually two times, it will be done again two times. Don’t immediately invest three days into it, start small and build up from that. Most of the time, only 20 minutes will do the trick.

Set a day for the “other stuff”

You should set yourself a day or half of the day each week for all the things you’re too busy to do during the rest of the week. At least for me, this is Friday afternoon. Office gets emptied sooner than other days, clients are calling less often, and I’m able to concentrate. It would be best to keep a list of small improvements (e.g., optimizations, report generation) that need attention but aren’t priorities on a current project or your client’s preferences.

Making small regular improvements is much more powerful than waiting for a carefree day and doing it all at once. The truth is that you’re probably never going to have a massive chunk of time to finish your side project or the “other stuff”. Even if such time were to come to our companies, we would surely find a way to waste it. 

Do it right the first time

It took me a while to realize that the time when I’m going to have a set of tasks, do them in the required time and go home is never going to come. This perfect balance when there are exactly 40 hours of work per week is probably not even theoretically possible. Almost every day, there is an exception to this rule. A client called; meeting took longer than expected etc. The only viable solution is to manage the chaos because it is not going to go away. If it does, the company is probably not growing anymore. 

There are rare cases where a workaround should be used because of a lack of time. Yes, you will frequently be pressured to do it quick and dirty. If you don’t want to encounter the same problem in three months, where you now have to change the hardcoded solution to fit another client or adapt the existing solution to be dynamic enough for multiple clients. If a scalable solution was done the first time, the second problem would barely exist. 

I have yet to see a situation where a piece of code which I wrote well for the first time (meaning it could be applied to more than one client, easy to read and debug) that it didn’t pay off in the long run. Sometimes I had to wait a whole year for it, but it always did!

A great answer on StackExchange inspired this post, and if you have something to add, feel free to write a comment or contact me directly via LinkedIn or mail.

Here you can find a second part discussing using Git’s solution to mentioned problems.