The return of the Computer Scientist

We hear a lot about the shortage of coders who master Big Data technologies. In fact, in the last decade, coders were just here to integrate various solutions together. Deep knowledge of the nuts and bolts of each standard was key, and a coder with many certificates from different companies was a very valuable asset. Recently, with mobile technology, we get the new generation of mobile programmers, in the same vein of the web programmers ten years ago. They can use the various SDKs very efficiently, have an eye for design, and can do fun stuff during a hackathon surrounded by free pizza.

With all these coders around, why is it so hard to find people who master Big Data technologies? Aren’t they just another bunch of coders? Not really. With Big Data, you need to write code with the constraints of a limited amount of hardware, with a growing difference in speed increase in processors, RAM and hard disks to process an ever growing amount of data. This is the time for Computer Scientists to shine again. We are back to the days when processing time is an issue and algorithms are key. Computer scientists were outshone by the casual programmers who mastered web technologies. They were asked to do such tasks as integrating five solutions from five different vendors and design specs for outsourcing the development of solutions abroad. But today, they are called to the rescue: shorter processing times, robust and scalable solutions, knowledge about statistics and data mining are in need.

Although some say we need DevOps: people who can write code that is production-aware and follow up with the production issues, I say we need computer scientists. They are trained to estimate the running time of algorithms, they know when to trade memory for processing, when to throw in some randomization and they have a solid background in statistics. They are here to reopen their books on Numerical Methods thinking on how to parallelize those algorithms on a hundred machines. With their background, they can bring down the processing time to do complex tasks or just tell you it is an NP hard problem and give you the choice of many heuristics to try out. Finally, they will know when database transactions are overrated.

It is a good time to be a computer scientist. There is no shortage of offers for people around us who are trained in algorithms, security, computation statistics, … For many years , because their skills were not needed for their paid jobs, they directed their attention towards open source projects, not with the intention to make any money, but to use their skills and gain peer recognition. Today all of us reap the fruits of these efforts and finally, after a long wait, computer scientists are back on stage.