Skip to content

Git, GitHub, and GitLab

CiviCRM uses git, GitHub, and GitLab to manage changes to the code. A solid understanding of these tools (especially git) is important for developing CiviCRM. This page provides some information about how CiviCRM uses these tools — but, due to the wealth of resources already available elsewhere online, this page does not attempt to teach you everything you need know about how to use these tools for CiviCRM development.

Tip

If you are new to git, a great way to get started using it within the CiviCRM community is to contribute to documentation. The editing workflow involves git in the same way that core coding does — but the stakes are much lower!

External resources

Repositories

  • GitHub - github.com/civicrm

    • As of 2017, most of CiviCRM's repositories are hosted on GitHub
    • Most of the repositories hosted on GitHub are owned by the "CiviCRM" organization.
    • Here are some of the most important repositories hosted on GitHub

  • GitLab - lab.civicrm.org/explore/projects

    • CiviCRM also has some repositories hosted on this self-hosted installation of GitLab

Git workflow overview

Whether you are contributing to civicrm-core or an ancillary project (using GitHub or GitLab) the process generally goes somewhat like this:

  1. Consider opening an issue on GitLab to describe the change you'd like to make. Not all changes need GitLab issues, but opening an issue is recommended if you are making significant changes, expect discussion, or expect your changes to be grouped into more than one pull request.
  2. Find the page on GitHub or GitLab for the project to which you would like to contribute. We will call this repository the upstream repository.
  3. Clone the upstream repository to your local machine. (If you are working on core, you should use civibuild for this step.)
  4. On the web page for the upstream repository, fork the upstream repository to your personal user account.
  5. Within your local repository add your fork as a second git remote. Learn more...
  6. Choose the correct base branch in the upstream repository as the starting point for your changes. (Usually this will be master.) Learn more...
  7. (If it's been some time since you've cloned) pull or fetch the latest changes from the upstream repository into the appropriate branch of your local repository. (You might also need to upgrade your civibuild site.)
  8. Create (and checkout) a new branch for your changes, based on the correct branch (chosen above) in the upstream repository. Learn more...
  9. Make your changes. (Take care to follow the guidelines in contributing to core.)
  10. Commit your changes. Learn more...
  11. Push your changes to your fork.
  12. Open a pull request. Learn more...
  13. Wait for someone else to review your pull request.
  14. If you need to make more changes later, commit them on the same branch and push your new commits to your fork. The new commits the will automatically appear in the pull request.
  15. If other people commit changes to the upstream repository which create merge conflicts in your pull request, then rebase your branch. Learn more...
  16. Once your changes are merged, delete your local branch

See also: reviewing someone else's pull request

Pull requests

terminology

The terms "pull request", "merge request", "PR", and "MR" all effectively synonymous. GitHub uses "pull request", and GitLab uses "merge request".

Creating a pull request

  1. In the web browser, navigate to the web page for your fork (e.g. https://github.com/myuser/civicrm-core ).
  2. Click Pull Request
  3. There will be two branches specified – the (first) left should be "civicrm" (i.e. where the code is going to). The second (right) should be your branch.
  4. Add a good subject and explanation, and submit.

To note: it's not possible to create a pull request for the civicrm-core repo on GitLab ─ use GitHub for this.

Writing a pull request subject

Pull request titles don't need to be identical to issue titles, and in particular, you may want to focus more positively on the changes in code than on the broader feature changes. Here are some guidelines for writing a good subject line:

When filing a pull-request, use a descriptive subject. These are good examples:

  • dev/core#5555 - Add "It's complicated" relationship type to defaults
  • CRM-12345 - Fix Paypal IPNs when moon is at half-crescent (waxing)
  • (WIP) dev/mail#67890 - Refactor SMS callback endpoint
  • (NFC) CRM_Utils_PDF - Improve docblocks
  • (REF) CRM_Foo_Form_Edit - Extract method checkFooBar()

A few elements to include:

  • Acronyms - You're welcome to use the acronyms below to flag your PR with certain characteristics.
  • dev/project#XXXX - This is a GitLab issue reference.
  • CRM-XXXXX - This is a reference to the now-deprecated Jira issue tracker. A bot will set up crosslinks between JIRA and GitHub.
  • Description - Provide a brief description of what the pull-request does.

Acronyms within PR subjects

You can put these acronyms at the beginning of your PR subject to flag it as such.

(WIP) - "Work in Progress"

If you are still developing a set of changes, it may be useful to submit a pull-request and flag it as (WIP). This allows you to have discussion with other developers and check test results. Once the change is ready, update the subject line to remove (WIP).

(REF) - "Refactor"

Refactoring is a technique of making small, behavior-preserving changes (see, e.g. Martin Fowler's Refactoring).

Because refactoring preserves behavior, it doesn't require as much scrutiny with regard to user-experience or product-scope. Rather, one merely verifies that the change preserves behavior.

Examples:

  • Extract a method or field
  • Pull-up a method from child-class to parent-class
  • Encapsulate a field

(NFC) - "Non-Functional Change"

Most patches are designed to change functionality (e.g. fix an error message or add a new button). However, some changes are non-functional -- they presumptively have no impact on users or integrations at runtime. The aim of flagging a PR as "(NFC)" is to streamline review on trivial changes.

Here are some examples and counter-examples of NFC:

  • Non-Functional Change:
    • Modify whitespace in PHP code.
    • Update a code comment.
    • Fix a typo or grammatical error in a help dialog.
    • (Maybe) Add a new unit-test where there was no coverage before (see below).
    • (Maybe) Update a PHP doc block (see below).
  • Functional Change:
    • Refactoring, e.g. replacing 20 lines of redundant code with a call to a helper function.
      • (Why? A reviewer would consider whether the helper is truly equivalent, better, or worse.)
    • Fix a typo in a symbol (PHP class-name, PHP function-name, HTML field name, etc).
      • (Why? A reviewer would consider dangling references to the symbol.)
    • Change the general wording of a help dialog or menu item.
      • (Why? A reviewer would consider impact on the user's comprehension.)
    • Add or alter an existing unit-test.
      • (Why? A reviewer would consider whether the change improves the correctness of the test.)
    • Update annotations in a PHP doc block.
      • (Why? Some annotations affect functionality, e.g. the @required annotation on an APIv4 param.)
    • Alter the build process.
      • (Why? A reviewer would consider whether the new build will work correctly.)

Pull request scope

A good pull request addresses a clearly-defined problem. There should be a detailed description logged in the issue tracker. Excellent PRs also increase test coverage. If you are tempted to do additional tweaks or code cleanup outside the scope of that issue, you could make a separate commit and include them in the PR if they are minor & non-controversial, or create a separate PR if they are more complex.

There is no size limit for PRs as long as they are focused on completely solving a discreet problem. As a practical matter, though, bigger PRs may take longer to review and merge. When possible, split "epic" issues into bite-sized chunks as long as each seperate PR is functionally complete and does not cause merge conflicts with your other PRs. In the latter case, add commits to an existing PR.

Reviewing a pull request

See How to review a core pull request

Who merges pull requests?

A person may be granted the privilege/responsibility of reviewing and merging pull requests who:

  • is an active contributor to the CiviCRM project
  • responds to communications in a timely fashion
  • is familiar with current CiviCRM coding standards and best practices
  • is a careful proofreader and tester, and who gives thorough constructive feedback

Git tasks

Cloning

When you want to set up a local copy of a git repo hosted on GitHub or GitLab, you clone it. Here are two ways:

  • Using the SSH protocol

    $ git clone git@github.com:civicrm/civicrm-core.git

  • Using the HTTP protocol

    $ git clone https://github.com/civicrm/civicrm-core.git

Using SSH is a little bit better because you won't need to enter your password all the time, but it does require some extra steps.

Managing multiple git remotes

Your local git repo is typically set up to track at least one remote git repo for operations like fetch, pull, and push. But it can be helpful to set up multiple remotes when contributing to repos which you don't own.

Common terminology:

  • Upstream repository - a repo hosted on GitHub or GitLab which you don't own but would like to contribute to
  • Fork repository - a repo hosted on GitHub or GitLab which you own and have created by "forking" an upstream repo
  • Local repository - the repo that lives on your local computer after cloning

Show the remotes which your local repo is tracking:

$ git remote -v                                                                          
upstream  https://github.com/civicrm/civicrm-core.git (fetch)
upstream  https://github.com/civicrm/civicrm-core.git (push)
myusername      git@github.com:myusername/civicrm-core.git (fetch)
myusername      git@github.com:myusername/civicrm-core.git (push)

The first column shown in the output is the name of the remote. You can rename your remotes however you want. Assuming your GitHub user name is myusername, the above output looks pretty good because we have two remotes: one named upstream (an upstream repo), and another named myusername (a fork repo). When you first clone a repository, git will set up a remote called origin which refers to the repo you initially cloned. In the above example we don't see origin, so that remote has been removed or renamed.

Read about how to use git remote to properly set up your remotes.

Tip

If you use hub, the command hub clone can help with this

Branching

Git uses branches to separate independent sets of changes. When creating a new branch, here are some things to keep in mind:

  • Choose an appropriate base branch
  • You'll need to keep your local branch until its changes are merged. Sometimes this can take several months. After it's merged, you can delete it.
  • Give your branch a good name
    • The name of your branch is up to you.
    • It should be unique among your other outstanding branches.
    • It should only contain letters, numbers, dashes, and underscores.
    • If you have a GitLab or Jira issue, you can use its number (e.g mail-111 or CRM-1234) as the name of the branch.

Create a new branch and switch your local repository to it:

$ git checkout upstream/master -b mail-111
  • upstream is your local name for the git remote which represents the upstream repository (e.g. https://github.com/civicrm/civicrm-core) to which you are contributing changes. Depending on how you have set up your local repo, this remote might have a different name, like origin or civicrm.
  • master is the name of the branch in the upstream repository on which you would like to base your changes
  • mail-111 is the name of your new branch

Choosing a base branch

When creating a new branch, you should explicitly declare a starting point.

Most of the time, your base branch should be master. In special circumstances, a patch may be accepted for the Release Candidate (RC), or Stable branch. The table below summarizes the policies for each branch.

Name Git branch (example) Version number (example) Acceptable patches
Master master 5.99.alpha1 This is primary target for most patches, including typical bugfixes, cleanups, and minor features.
Release Candidate 5.98 5.98.beta1 Fixes for critical, recent regressions. The regression should be traced to a specific, recent change. In a typical cycle, only 1-10 RC patches are accepted.
Stable 5.97 5.97.1 Backports of fixes for very critical issues. This is usually only done by the project maintainers. Make your PR against the RC instead and they will backport if necessary.

If you are unsure about which is the current RC or master branch, you can refer to https://download.civicrm.org/latest/, and look at the numbers in brackets to the right of the filename, e.g. if you see civicrm-RC-drupal.tar.gz (5.24.beta1-...) it means the current release candidate branch is the 5.24 branch.

Don't make multiple PRs against multiple branches for the same thing even if the bug is present in both versions, e.g. both the RC and master. Consult the table above and just pick one branch.

Committing

As much as possible, separate your changes into distinct commits that each make sense on their own. Using smaller commits will make your changes easier for others to review. You can clean up your commits once you have finished getting your code right.

It is often helpful to put your commits in separate smaller PRs as a reviewer might spare 30 minutes to review a function extraction of a variable cleanup and merge the PR fairly quickly but if you put 5 in the same PR they might not be able to find a block of 2 hours to review them all and it might languish. Doing preliminary cleanup PRs while you are still working on a problem can be helpful as you are improving the code as you go rather than creating a huge amount of change that will be hard and/or risky to review. While you are still working on a problem is also a great time to start adding tests for the code around it to core.

Writing a commit message

When making commits, remember that this isn't just a small personal project: your audience is hundreds of other developers — now and ten years from now — as well as end users trying to understand features and bugs. By leaving a commit history that makes sense — both in content and in commit messages — you will make the code more legible for everyone.

Follow these guidelines to write a good commit messages:

  • The first line should be a meaningful subject, which should:
    • be prefixed with a GitLab or Jira issue reference (if the commit is to CiviCRM core)
    • mention a "subsystem" after the issue number
    • be 72 characters or less, in total
    • be in "Sentence case"
    • use the imperative mood
    • not end in a period
    • examples:
      • dev/core#999 - Civi\Angular - Generate modules via AssetBuilder
      • CRM-19417 - distmaker - Change report to JSON
  • (optionally but recommended) After the subject, include a short body, which should:
    • have a blank line above it (below the subject)
    • be wrapped at 72 characters
    • explain what, why, and how

Rebasing

Sometimes when you make a pull request someone else merges a change into the upstream repository that conflicts with your change. The best way to resolve this conflict is to rebase. It's a good idea to read about rebasing so you understand the theory. Here's the practice:

Note

In this example we have two remotes set up:

  • upstream which tracks the upstream repo
  • myusername which tracks the fork repo

Also we are working on changes in a branch called my-branch

  1. Update your local master branch

    $ git checkout master
$ git pull upstream master

  2. Checkout the branch that has conflicts you'd like to resolve

    $ git checkout my-branch
$ git rebase master

  3. See which files need attention

    $ git status

  4. Make changes to the files which resolve the merge conflicts

  5. "Add" the files to tell git you have resolved the conflicts

    $ git add CRM/Utils/String.php

  6. Continue the rebase

    $ git rebase --continue

  7. Force-push your changes back up to your fork

    $ git push -f myusername my-branch

  8. Now, if you go to back to the page for your pull request it should no longer show merge conflicts

Rebasing for cleanup

When you are working on a PR you will often make lots of commits in order to get to the right change - these are your workings. However, when we merge we want the commit history to show the final change - not the workings so we will often ask you to rebase. It's fine to have multiple commits in a PR but each commit should be complete in itself and a logical change in itself. If you make a mistake in one commit it shouldn't be fixed in another (unless the mistake is already merged).

The trick to a cleanup rebase is to do it in interactive mode. The process is similar to above and it's usually easier to do the above first and THEN the interactive rebase to make sure you have a nice clean start before starting your interactive rebase.

Assuming the upstream repo is called 'upstream' the command is

git rebase -i upstream/master

You then get to squash or remove commits are fix up the history. Do a force push (per above) when you have finished