Lőrinc Pap: Hello Venkat, we would be very interested in your take on programming in general, and on functional programming in particular.
Venkat Subramaniam: Well, first of all, thanks for having me here. It's a pleasure to talk to you and to your listeners. So, programming is something that is life to me, in a lot of ways. I've been programming for a few decades now and, often times, people ask: "How come you're a programmer after all these years?"
A lot of people got into programming because of science and math. As an engineer, that's what I also wanted to do, but, over the years, what really kept me as a programmer is actually art in programming.
If somebody writes a poem - maybe a love song - well, there's not only one poem to write. You could write an infinite number of poems. In a lot of ways, writing code is a lot like that. You could be writing code in so many ways, having exactly the same result. And that's one reason why I really like functional programming. It is a very different form of expression and it is very concise. It is something that is very expressive and elegant, and it conveys the idea without dragging you into the details. Imperative style programming is really loaded with accidental complexity, so when we start programming in an imperative style - as we have done for decades -, we are really going through a lot more effort to do the same work, compared to functional programming which removes a lot of these accidental complexities. It's the art and programming, combined with the ability to convey the idea that the machine must do what it's supposed to do. This is what really incites me about programming.
I've noticed that in many of your tweets you talk about your intuitions regarding code. You don't subscribe to the notion that, if people follow given instructions, they will get better. You mostly optimize for testability, for maintenance, for expressivity and not necessarily for speed.
When you deal with a complex system, one thing we come to agree upon is that change is the only constant aspect. One of the biggest challenges is that, the minute we write software, the next thing we want to do is change it.
Software is a non-linear system: you change something here, something would break over there. I have a very strong desire to think about and to improve the economics of software. I want to create stuff in a sustainable way. That's one of the reasons why I am a really big fan of going through automated testing. It gives me very rapid feedback: what worked yesterday, also works today. When I'm covering my back, I can go wilder, be more creative and try new ideas.
Just to give you an example, this happens quite often. I was teaching a class today and we've had automated tests already, and then we were writing code to pass those tests. I implemented a solution and one of the developers of the class said: wait a minute, you could actually write this in a slightly better way. Rather than argue, I immediately changed the code, and the tests ended up passing again. This way he immediately gets the feedback that his idea is actually not a bad one. You don't have to guess. You can know that it's working right away. To me, that's extremely positive. We don't have to guess, let's go ahead and try it.
The time and effort put into code is really valuable. It has to be really cost-effective. It's why I would rather build fewer features with better quality, than more features with poor quality.
Venkat Subramaniam and Lőrinc Pap
How do you think this journey should start? Should kids be taught to program? Should it be functional from the start? Should it be TDD and Agile from the beginning? Should laymen start to program as a hobby?
It has been an amazing experience for me to watch children learn. We have to keep one thing in mind: when it comes to learning, we cannot compete with children. As adults we have to put an effort into learning, while this is a way of life for children.
It's important to let them learn the right things to do. Caring and passion are the first things they should learn. Once they are passionate, they can look at things creatively, and they can produce value in a sustainable manner.
When my son learned to write code, in one of the projects he worked on, every single piece of code he wrote was automated and tested - and it's the only way he did it.
I told him "I'm not going to sit here and write the code for you, I'm gonna teach you the basics and walk away. You're going to build the application yourself."
When I teach kids these days, I actually give them automated tests and tell them to write code to make those automated tests pass. And some of the kids and parents come back and say "it's actually pretty amazing to do something and see it actually work and see the results so quickly."
I do believe it's important to teach them a sense of quality, a sense of feedback loops.
They quickly realize that what they type is not going to work all the time: our brain is very capable, but to err is human. We are really good, but we have to bring in things that compensate for the areas where we are not good. We cannot remember every meeting we'll have - that's why we have calendars and reminders. It's the same with coding: we cannot remember every possible permutation, that's why we have automated tests.
That's what we'll have to tell kids also "you are smart, you are capable, but there are things you won't be able to do as a human, and there are tools for you to leverage - and I think it's extremely important for us to be able to teach them that."
It was fascinating for me was to really let them learn, without pushing them towards jargons. I was teaching Javascript to children recently. I asked "how many of you play music" - and all hands went up. It was an aha moment for me: when you have a music script and you're pressing these keys and the piano makes this beautiful music, as a person playing music, your job is to read the script and make this machine do stuff - to interpret it.
Programming is like that: it reads the script and moves the machine, so that it does whatever you asked it to do. The kids were like "ok, that makes sense."
I think the key is to really bring it down to the level where they can actually appreciate and realize that this is not really hard stuff, it can be fun stuff to work with.
What do you think is the importance of functional programming in this area? Should kids be taught functional programming from the start? Should they just arrange shapes in different ways, or should they program directly in a programming language, as adults do?
I think that kids should learn problem solving first. I think we're going in the wrong direction if we focus on programming alone. Programming is really only a small part of what we do: the major part of is problem solving. Programming is a vehicle, a tool: we have to help them get really comfortable solving problems.
The other important thing is to teach children, very early, that there isn't only one way of doing things. What many adult programmers get entangled in is that they feel that there is only one language, one solution, one way of doing things.
If, however, you look at programmers who are better than others, what do they do? They are open to ideas, open to suggestions, open to looking at other solutions.
I teach at the university as well. They aren't kids as in children, but not all of them are grown up adults either. One of the things I really try to help them see is: I want you to learn 3-5 different programming languages, so when you come out of the course, I want you to know that there isn't only one single way to do this. When a problem is given to you, I want you to openly think about 3 or 4 different solutions. I think that's what we should be aiming for in children as well, so they realize that, when given a problem, they have to be open to looking at multiple solutions - whether it's functional or imperative style programming, dynamically typed, or static language. I don't want them to dig themselves in a hole and say: this is the only way of doing things.
This is the time for them to explore and figure out that there are actually multiple ways of solving problems. You must let their mind grow into a thinking which is a diverse set of knowledge.
There is a very old research, about natural languages, called the Sapir-Whorf hypothesis: as a human, what we think is greatly influenced by the languages we speak.
I definitely attest to that: in programming languages, the design that I can think about is greatly influenced by the programming languages I know.
I used to know only a single programming language, but when I started learning others, it was eye-opening: wow, these are the design ideas I never even thought existed, because the language never gave me that thought process.
Knowing these different languages expands your design. You realize that when a problem is posed, you should really consider what your options are, and maybe select a language or library or tool, based on the problem.
To me, as a teacher it would really be unfair if I tried to drive them in one direction - you have different options, you have to explore those.
How would you convince people to try out different languages, e.g. `you should really try out Haskell - I know that you're scared, that it's an ivory tower language, only used in Academia, but try it yourself and see instead`, or maybe `Clojure is dynamic, it's a Lisp, but give it a shot, you'll be surprised'?
When you're exploring a certain concept or idea, you want to go back to the roots. When you want to learn a particular concept, you want to go to a language that does it really well. Why should you learn another (esoteric) language? The reason is that, if I know different languages, the time I need to learn another one decreases.
When the mental "muscle" has become lethargic, I keep doing what I'm comfortable with - when a new thing is thrown at me, my mind doesn't respond to it.
Learning is not easy, it takes a lot of time and effort. However, it took me about two days to learn Groovy. This is not because I'm smart, but because I have been programming in Ruby for a few years and all I had to do is asks the question: how is Groovy different from Ruby or Java.
When C# introduced the yield keyword, my question was: how is yield in C# different from yield in Ruby, because I've used yield in Ruby and I know these are generators. Javascript recently introduced generators and you have the yield keyword in there. Ruby is so different from C#, which is so different from Javascript, but you are able to relate these ideas. The way you learn becomes fast when you do that.
I have been learning Erlang for a few years, for absolutely no good reason, but then I had to learn Scala very quickly, and I said: Scala has actors, Erlang has actors. While Erlang is a dynamically typed language, Scala is a static one, but it doesn't matter. I ended up learning Scala very quickly.
We don't learn in huge chunks, we learn in deltas. This is why it is very important to learn different languages continuously.
As I tweeted: "the time you need to learn a new language is inversely proportional to the number of languages you've learned" (https://twitter.com/venkat\_s/status/8301627048).
To answer your question - what's my motivation to learning something different -, you end up learning things very quickly. The more you already know, the more diverse your knowledge is.
It expands your design skills, and your ability to go and explore - it keeps your mental "muscle" active.
As the Pragmatic Programmers say: you have to learn a (very different) new language every year. Forget year, go on and pick up the next language.
How would you convince those who don't see the need for any new language. They already have enough problems in production, and the company won't let them use another language: why should they bother learning it then?
I would like to reflect back on my carrier.
In every single company that I worked in, the language that I used was not the language or the environment my next job was in. While I produce value, and give results to companies, I got to build my career too.
When I worked on a job requiring C++, I noticed there is this new language, called Java. However, my company did not abandon it and would not start programming in Java overnight. Therefore, this is what I did: during the day, I worked in C++, because that was my day job, but, in the evening, I would spend an hour learning Java. When you do this, you will notice job openings that require this language, or your old company comes to you and says: we are considering this new language and looking for people. Moreover, this is where the word `professional` comes in. What would your reaction be if the doctor said: I want to treat you with the medicines I know. Don't bother me with these new medicines; I already have enough things to do. You want your doctor to say: there are some new tools on the market. They help you with this particular problem, we should try it. I'm the novice, but I want the doctor to discuss my options.
In my job, I'm that professional. Therefore, when my company says "I want you to solve a problem", it's my job to respond "here are the solutions".
As an example, I had a company call me and say "come and train my team on this particular technology". They have a highly complicated system with millions of users on the market, so very high demand from concurrency.
The manager asked "Venkat, do you know why you're here?" He takes me to this other room where he presents me two guys: "these guys have seen your talk and said: 'wow, this addresses concurrency in a lot better way'". Over a two week period they developed a prototype and said to the rest of the people "you can continue to use this concurrent solution" - everywhere you turn, there's locks and synchronized and deadlocks and livelocks - "or, we did this module and reimplemented it; there are no locks in this, no concurrency issues. It's up to you."
"So we evaluated it and it was clearly the sign that we're not going to succeed in what we do in the way that we're already doing: we have to change direction! So,Venkat, this is why you're here: we have decided to change direction and these are the guys who did this for us. They would not have done it, had they not taken the time to learn it and to go give it a try."
How often do you see that companies are willing to abandon work that they have done previously in favor of upgrading to a new paradigm?
It's not too common, but it's not unusual either.
You and I should talk about technology, but, at the end of the day, we must explain why it's better for the business. The wrong thing to tell the business is: this technology is cool. We have to speak in terms that they understand. So go to them and say: How will it affect the bottom line? Does it save them money? Does it get them faster to the market? Does it help them develop products with fewer bugs in them? Does it help them speak in terms of business value.
People often ask me "how do I convince my boss e.g. TDD is important?" And I always ask the question: "Are you convinced? If you are not deeply convinced, you cannot convince other people. Have you done your homework? Have you already worked on it? Have you created a prototype? Once you have, put it out in a way, they can see the value."
In a way, we have to be salespeople as well, not just technology people. They don't care about the technology. They care about the results, the benefits, so we need to express it in those terms.
Being a consultant, what do you see most often as the lowest hanging fruit, i.e. the biggest problems they have, that can be resolved with the least effort?
When I go to companies, they are often very reluctant. They've been having these problems for a while and look at me like "so you think you can come and solve these problems, ha?"
Recently, I went to a client and the developers didn't want to smile, they didn't want to shake my hand - they've been having this production issue for 6 months. The manager says "here is this guy, Venkat, to help you in solving this problem." And they are like "who are you, why are you here?" I said "guys, I'm not here to fight with you, you are the ones who were working on this day-and-night. I'm just here to learn from you, I want to help you." So, we sit down and struggle together and they're getting angry because you're asking a lot of questions "take it easy, I'm the guy who doesn't know anything, help me in seeing this, remember?"
In every project that I helped with, there's one common thing developers often don't do. When I walk in I'm able to solve the problem, not because I'm smarter, but because I'm distancing myself from the problem - I'm isolating the problem from the surroundings.
You have this massive enterprise system where things are not working. You're frustrated and you have to sit there and say "let me try to zero in on the problem." One of the techniques we have to learn as developers is how to sift through these things and say "no need to work here or there, let's rather focus on this part." You can use tests to help you in focusing on the problem, but in systems where there are no automated tests you may need more creative ways to help in isolating the problem from its surroundings.
This goes back to the topic we've discussed earlier: how do you teach children how to solve problems, because that's what's so critical. In this environment, it's not about the programming language, it's not about the tool library, it's about the inability to solve the problem.
You are an agile and a functional programming advocate. What would you say the advantages of these ideas are?
When companies adopt agile development, they should stop and ask: Why are we doing this? How does this benefit us? We easily go for the ceremonies, the mechanics, the rituals, and we lose sight of why we're doing this.
For example, the problem with the Waterfall technique is that there isn't a good enough feedback loop in it. After going through requirements gathering, analysis, design and finally getting to programming and testing, if something doesn't work (since you've spent so much time doing it), you will start to panic. It's so late in the game, that there's not much time to do anything about it.
Agile development is feedback-driven development. When I go to a company and they will say "we're doing agile", then they tell me "we've fixed the deadline, we've fixed the scope, but we're Agile". One of my most popular tweets ever is related to this: "I've set the wedding date. I've not asked her out yet" - this is how many software projects are managed https://twitter.com/venkat\_s/status/617052918491918336).
If we're not doing adaptive planning, then what is left to be Agile? We have to adapt what we do based on the realities of the world around us, based on what changes around us. Planning is so important in fact, that, in Agile development, we don't do it in a big phase, we do active, adaptive planning every single day.
We have to be realistic; if I say "I want this result, I want process, but I'm not going to change anything, it's you guys who should change things" - that's not Agile.
We have to stop and ask ourselves "am I really developing based on frequent feedback or am I just using the word Agile to feel good", "am I being adaptive, or am I being adamant?".
If I'm a developer in a company and I'm the only one who wants to do agile, how can I be the change that requires this transformation?
First, you should be non-threatening; people don't want to work with you if you are. When people see that you are really genuinely trying to improve, there'll be more willing to work with us. The second thing is to really help people grow.
I mostly never had a boss, who said "You will do this! I'm your boss, I will tell you what to do, and you do it." I could always walk into their office and say "I don't understand why we want to do this. If we do this, you're saying we're going to get this result, but here's why I don't think we will get that result. Here's why we have to change it." It's not that they will agree and do everything I tell them to, but it was very healthy to go and talk with them, to challenge them in a constructive way. I think it's important to grow an organization with a culture where anybody can come and talk.
From day one, the culture at Intel was that you can walk up to the CEO and say: "I don't agree". That's what makes a great organization, to challenge everybody in the organization and say, "we can do better". We've got to question what we do, that's the culture we have to build.
You think there's a better way to do? Sit down next to one of your colleagues and say, hey, let's try this, let's explore this, and you're going to have people buy into this, slowly, one person at the time.
People don't want to be told "this is better to do", they want to be shown. People may not believe what you say, but people see your action.
Success is contagious. If you tell me you'll be better, I'm not going to believe it, but if I see you are successful, you're releasing code with better quality, you're going home, you're not fighting those bugs in the evening… What is it that you do, that I'm not doing? I want to come and see what you do. There is nothing as convincing in an organization than success. There are people who come to me and say "How do you convince your boss there is a better way to do something?". You go and say: -"I've got this wonderful idea and I want to do it" -"Sit down, go back to your desk and get your work done"; but when they see you're successful, they say "what can we do to support you, how can we facilitate you?" - I've seen this personally in our organizations, when you are successful, people want to do everything to make you successful, they're like "stay out of his way, he's succeeding" and there is absolutely nothing better to convince people than your own success. So go, do stuff you believe in, succeed with it, people will take note of it and say: "let's go do this".
Could you give us examples on how you became successful in these areas? What were the techniques that you tried - like Agile or functional programming - that made you (and may make other people) successful?
First, we should be willing to fail early, fail often. Anything you do, just look back at your life. The first time we had a meal, an adult had to feed us. We learned how to eat. The first time you started riding a bicycle, you fell; you had bruises on your knees, and then you started learning how to ride a bike. The first time you started driving a car it was the scariest day for the city. Or maybe you remember the first day you got in the car and asked that girl out. We have to try; we have to improve by doing stuff over and over and over. The sooner we do, the sooner we try. The sooner we fail, the sooner we can get better at it.
The second thing I'm gonna say is: the worst way to do anything is to ask for permission. I've learned this lesson the hard way. I went to my boss one day and I said: "I've got this great idea, I want to do this" and my boss said: "Sit down, go back to your desk and do it, do this work, we don't have time for this stuff, sorry we got other things to do". I realized this is so important to do, but why wouldn't people want to change the way they do things? I started my own circle of influence. There are things I can change; I don't need permission from other people to tell me. Who needs to tell me that I need to write better quality code? - Nobody! Who tells me that I have to create better quality variable names? - Nobody. I can create better variable names, I can create better function names, I can write automated tests for my own piece of code, I can write smaller methods. You start changing things in your sphere of influence or of change. When I start doing it, other people begin to take note of it and then they say: "Hey, that's much better, I want to do it". Therefore, one of the best ways to affect change is to actually start doing it. It gives you the optimal experiment; it gives you the possibility to try things out. Not everything I do is going to succeed; not everything I try is going to succeed, but that's perfectly ok. Then you figure out your way around, because you tried so many different things. My first recommendation is: go try it, go to your user group, go to a particular presentation, go to a conference, and you hear something different, something you've never tried, you tell yourself "I'm gonna try it". It could be a small project, it could be a home project, it could be a side project, but I want to try it, because when I do, I fail, then I learn, then I get better, and then I've raised my level of competence because of trying it. Everything we do comes from trying. There are some wonderful, very, very capable people, and they often don't succeed, not because they are not smart, but because they aren't trying hard enough.
In what direction do you think the industry is heading? What other promising methodologies do you see that could make coding and communication better?
Almost everything, every change in technology has been caused by business change. Technical people can sit down and say: "We are technical people, we affect change". No we don't. It's the business that drives us, rightfully so. When did we adapt to optical technology? When did we start adapting to things like Ajax or things like Rich Web? Things are changing again. We see Big Data, companies want us to build responsive systems, so technology has to keep up with these demands. Therefore, our efforts in function programing are driven by the market demand, concurrency, responsiveness, more complex applications, more concurrent applications. Technology is there, but business drives technology.
Where is the future heading? Right now, we see a demand in big data; we see a demand for mobile devices. One thing I can certainly tell is that the future is not going to be in desktops, the future is going to be not just in mobile devices, we are talking about embedded devices. When people talk about wearable devices, forget about watches. We are talking about nose ring and ear ring and tongue rings, all communicating. So, I repeat, this is very powerful, but we haven't even realized what that is in terms of its magnitude. We are talking about nanodevices, and I think that's where the explosion is.
This is a beautiful time to be in the field. I think the way we will be computing will be very different in the future. The biggest hurdle for us is trying to frame the future with the limited knowledge we currently have. We are going to have leaps and bounds of changes we wanna do. The key is to append what is holding us back and start doing things that require us to change the future. I think that, in the next ten years, we will see things we have never imagined. If you really look back, every five to seven years we are looking at technologies that did not exist. How could we currently say: "In the next twenty years we know exactly what we're going to do." The key is not to predict the future, but to adapt to it. The biggest win is to adapt to the future. Darwin did not say it's the fittest which survives, but rather it's the most adaptive which survives. As a developer, my success is not in predicting the future, because the chance of predicting it correctly is so low. I used to say: "I don't care what the future is, but I'm gonna be there to respond to it".
by Ovidiu Mățan
