Being a Researcher and an Engineer

What exactly is the difference between Research and Engineering in computer science? I've been reading a lot of different opinions about this topic, and nothing quite ever clicked for me. Lots of it was abstract without any hard details about what life actually meant in research vs engineering. While there are symptoms about what the differences are, product groups have time lines and customer bugs whereas research is protected from such work and research occurs in a bubble with a tech transfer back to product groups, there wasn't a very clear difference in what it meant to be a researcher versus an engineer.

After much time in both an industrial research lab at Oracle Labs, a Ph.D program, and working as an intern at Adobe Labs as an intern, I have some vague ideas about what the differences are. I think the biggest difference philosophically, is that the researcher must like the journey of discovery more than they like the sense of accomplishment.

Research is by definition, is something new. Research doesn't have to work, it doesn't have to be practical, it just has to be new. Taken to the logical extreme, imagine a new piece of knowledge that never existed before but has zero practical use. Great researchers have to not only be aware that this is a possible outcome, but has to be satisfied with this outcome. A researcher in the pure sense loves knowledge and the pursuit of it, for the sake of more knowledge, and not necessarily to accomplish a goal. Someone who reads wikipedia and clicks through every article with ADD is a fantastic example, because they love learning just because. Figuring out why things work, why things don't work, when things work, and what would be cool are traits of a researcher.

Engineering on the other, has to make something that works. In the real world, it means using less sexy technologies based on proven techniques and with a very high chance of success. The number of unknowns are constrained quickly to build things on time that work in most applications. Hacks that work are prized over elegant theories that might work. The end result and goal is more important than the process of figuring out how to solve the problem, although both are important. I think engineers are motivated more by admiring something that worked than necessarily the steps of how to get there. I've met a number of these people, the work horses who don't care what you want done, but they will get it done.

I don't mean to say that these two distinct camps cannot co-exist, and I don't think any single person sits at one extreme or the other. Everyone sits somewhere in the middle of the spectrum. I think the best research and the best engineering actually merge at the top. The best products both contribute something new as well as create something that works magnificently - From everything I've read, Tesla and SpaceX hit this target. The first iPhone probably required both large amounts of manufacturing research as well as engineering knowledge. The two sides met at the top to create something extraordinary.

However, this creates a conundrum for someone in the middle. What if you are equal parts research and engineering? There has been some discussion about this in other blogs, the concept of the research engineer, the applied researcher, or software engineers doing advanced development. These phrases are describing the person stuck in the middle, someone who knows how to do both research and engineering, but isn't quite comfortable in either camp. They love building things that work, but also have some ADD to build things that might never pan out. They love learning new knowledge and creating new knowledge, but also shoot for a goal with a high chance of success. I think most industrial research labs are moving towards this model, with Google leading the way.

If research is walking towards engineering, is the converse also true, that engineering is moving towards research? I think in Computer Science, distributed systems engineering is moving faster than research is. In many large scale internet companies, they have the resources and need to make tens of thousands of systems work together, something academia could never afford. Their end goals are driven by business needs and engineering is leading the way, probably with input from researchers. In that sense, my naive observation is that engineering is actually driving the field more than research is. I also get the sense that overall, engineering contributes more towards advancing the field than research does.

That's a very bold claim, that engineering drives the field more than research - the ideal notion would be the other way around. Research comes up with innovative new solutions to problems engineers couldn't quite fix yet. I think the problem is exploring a solution space - research is great at exploring a solution space, which means it comes up with a large amount of solutions that don't work. I think one of the best things I learned at Oracle Labs is from the research director Eric Sedlar. The conversation went something like:

Me: If we have a dead horse, why do we keep beating it and not try something else?
Eric: If you have a dead horse, you should study why it died so that we don't kill the next one.

Enlightening huh? The problem is, that like depression and happiness, there are a couple of known ways to be happy whereas everyone is depressed for a different reason. There are no journals or conference papers on approaches that don't work, only conference papers on techniques that do work. Nobody celebrates a dead horse as much as a horse that's winning. It's true that learning why a project failed is important, but few will celebrate it. Hence, a researcher has to like the knowledge more than the goal.

Engineering on the other hand, constraints itself quickly to techniques that are known to work. Building something that works is prized more than using new technologies to try something new. There is a business problem that needs to be solved and time is spent solving the problem in a quick manner.

After all this philosophical talk, is there a test you can take to determine if you're more of a researcher than an engineer? The only thing I can think of is would you rather learn or invent something cool that nobody cared about or something that works and solves a problem you have? Preferably you can do both, but if you had to pick one extreme, which would it be? I'm not sure it's the best test, but at least it's something to chew on.