If you are reading about continuous integration and are working on teams that use Pull Requests, Merge Requests, and these two things are not fitting in very well, this video is for you.
Make sure to enable the English Captions if you’re not a Portuguese speaker.
The text below is a transcription of the video.
When people talk about continuous integration, we have to understand that there are two ways to understand the term continuous integration.
There is the first way, which is to understand continuous integration as a way of speaking: look, I have a server that runs all my tests, or runs a build procedure for each commit, every time I put a new code in a repository of code, and this test automation runs independently of my machine, that thing of running tests and saying on my machine works.
No! I have a machine or a set of machines responsible for running these tests ensuring that the tests are automated or the build routine if you need to generate a package if you need to see any artifacts if these routines are working properly and give you feedback that everything is fine, that everything is working, and you have this Continuous Integration server.
This is a term used, for example, in Circle CI, which is a tool that is “CI as a service”, we even use that term, because it is a tool in the cloud that is running our suite of tests every moment. This is one of the meanings of the term continuous integration, having the tests running in an automated way and you getting feedback on that.
The second way to understand this term Continuous Integration is to understand as a software development methodology. A methodology is a way of working in teams, a way of working in software teams. This methodology was very popularized by Martin Fowler, who has a book. The book is super famous and in it, the author explains to us how we should work in order to have the benefits of Continuous Integration.
One of the things that the Continuous Integration people say is for us to always integrate the code at least once a day. What does that mean? It means that we would be working in a team of several people and every day when developing a feature, you would download the code from the centralized repository, work a little on that code and, let’s imagine that you are done, have run the tests and they pass, you would create a commit or more than one and then you would integrate this piece of code in the centralized server to be distributed to everybody.
Then you would run a git push and you would post your changes and eventually on the same day, or the next day, the people and other people who work with you would download this new code that you put in, so it would be every day, at least once a day, you would be running git pull and git push, working on the changes of everyone on the team.
if you think about it, it doesn’t fit very well when we’re working with Pull Request. Usually, we create a separate branch for our feature. You will write it, write a piece code in a kind of isolated way.
Often you will not be able to finish the branch, the functionality, on the same day. You will finish it maybe the day after. It is important that it does not take too many days, because this will bring other problems when it’s time to merge the branch, but you end up working in isolation, kind of isolated, and then when the code is working, the tests are working. You have tested on your machine that the functionality is ok.
You create what we call a Pull Request or Merge Request, in which you will put your piece of code, the code that you modified, you will put a description of what you were trying to implement, the paths that went right, the paths that went wrong, and put some of your rational there, and a discussion will begin, a nice discussion in which you will ask for feedback from the rest of the people who work with you on variable nomenclature, on the architecture of code, whether the code is clear or not, its clarity.
All this to improve the maintainability of this code. If you do this, if you create this branch to transform it into a Pull Request afterward, you are not integrating code every day, you are not running “git push”, you are not running “git pull” or “git push” directly to the master, and it’s a different way of doing things.
I think when you throw code directly on the master, in this git pull and git push dynamic, I don’t think it’s cool because you end up, in a way, getting the master dirty. I like to keep the master always in a “deployable” state, in a state that we can take the code and put it into production. And when you do it, when you use this continuous integration technique in that traditional way, I think you end up putting some unfinished code, code that sometimes doesn’t have decent test coverage, code that you haven’t tested correctly on your machine or not yet thought about possible implications.
And what happens, in many cases, people end up putting, to avoid having to put this code in production, people end up putting a technique called Feature Toggle, that you put an “if” in some parts of your code to hide that functionality that is not yet finished, that is still being developed.
So you are hiding your work in progress through an “if”, through a Feature Toggle that will be turned on or off according to some rules. I much prefer working with Pull Requests, working with branches, short branches, ok? Branches that are not going to take 10 days, 15 days, months. This is not the idea. The idea is that you have a deliverable to your team that has not only the modified code but also the context in which that code happened, why that code exists, is it because of a feature? it’s because of a bug? what have you tried to do? what did go right? what did go wrong?
So like this: code, context, and discussion. The discussion is super healthy for you to have feedback, for you to disseminate knowledge within the team. Of course, this discussion needs to be healthy, it needs to be done in an inclusive way, in a non-offensive, non-violent way (for those who know the non-violent communication book). And this way of working for me is far superior to continuous integration. And yes, I mean that these things are not so compatible.
I think that, if you take the definition of what the continuous integration is, it will be very clear what is the work methodology you have to follow and if we go to this Pull Requests dynamic, based on GitHub, which made it very popular that through open source projects, we are copying this way of working from the open-source into our commercial projects or within our company, our group,
I think these two techniques are in conflict. And I prefer the Pull Request technique of the Non-Continous Integration. The joke I make is that this technique is a Discrete Integration. It is a very playful term to discern this difference well.
And giving a historical context, which I think is important, the continuous integration is from the beginning of the 90s. So it has been more than 20 years since these concepts were documented, they were used more. Since then, we have had important creations in how we develop software. To start with, git came up, and I think that’s the main difference to keep in mind.
At the time of Continuous Integration, it was still in the early 90s, the teams used relatively archaic source code managers (SCM). At that time you had subversion. Before that you had CVS. Some teams also used commercial SCM, you had Source Safe, and you had several others as well.
When git came up, it brought up this idea of branches and merges, which in other SCM was extremely hard to do, it was very difficult.
In Subversion, when you created a branch, you had no intention of merging that code afterward. There were a lot of conflicts, it was very difficult. It was something that you created a branch to save any idea or release. Not at Git.
We have this, this good practice of creating branches then merging it. You have much better tooling than at that time. And of course, GitHub also appeared, the Github that popularized this Pull Request idea, which did not appear on GitHub.
This idea has been around since the development of the Linux kernel, where you had… when Pull Requests were done through the email list. You would send a diff attached to the email, would send it to the mailing list, the one interested in merging it, the delegate, using those very specific terms of kernel development, they would do the pull. They would pull the diff for them. That change that was in that diff would be merged.
What is cool is that at that time there was such a discussion that I mentioned, the context. When you would send the e-mail, you would send the diff, would send the context, what you were trying to do, the implementation of something, and such. And in the e-mail list, there was a discussion about whether that code was acceptable or not, why, what had to change. And there are even the famous Linus emails.
He was very aggressive, very rude to people. He has even apologized to people, changed his position, but this idea of the discussion already existed before GitHub.
But it is clear that GitHub popularized this technique a lot, and brought this vision of the open-source world of how to work within teams, which everyone knows. And I think Pull Request brings collaboration, it brings a way to spread knowledge much faster, very nice to other teams, and it doesn’t matter if you’re a senior or if you’re a junior, you do have to accept code feedback, give feedback to others, no matter what the difference is between who is writing the Pull Request or who is commenting.
I think it is worth it, yes. Every team should have a well-structured, very cool, very inclusive code review process. And of course, we add Git, we add GitHub, these are new tools that we use to develop software
I think that continuous integration ended up getting a little behind. We need to update, modernize it. It was very much based on the problems of 20 years ago that made perfect sense, but now we need a different path.
So a joke I always tell: I’m pretty sure you, or your team, don’t use continuous integration, because you’re probably using Pull Request. You are collaborating on feature branches, on branches for functionality, and thus you are not following that methodology of continuous integration. I’m pretty sure that you don’t use continuous integration.
And what’s better, there is no problem!
You shouldn’t even be using it, in my opinion. I think most teams should use code review. Of course, it has its exceptions, and the teams that are being built, Startups, Startups in growth, benefit a lot from having this collaboration process, code review, and Pull Requests.
Within the continuous integration, as an update, a new term has come up, Trunk-based Development, that is, development based on the main trunk, in the main branch of the SCM.
This is a cool discussion. But for another video. If you liked it, subscribe, and we’ll see you in the next video, OK? Give a comment if you liked it if you agree with me. I will be happy to hear your opinions. OK? Thank you so much, folks. See you in the next video.
[wd_hustle id=”9″ type=”embedded”/]
Co-founder and CEO of SourceLevel. Previously co-founder of Plataformatec.
Loves to talk about software engineering, Elixir, coffee, mechanical keyboards, multitools and his 3 big dogs.