These review standards provide a name and description for common review tasks.
When reviewing a pull-request, you may consult this list for ideas/inspiration on things to check. If you find a problem or feel that some QA task remains to be done, then it can help to post a link to the relevant guideline. This practice allows newcomers to understand the critique, but it doesn't require you to write a long, bespoke blurb.
Each standard has a code name (e.g.
r-explain). These make it easier to reference the standards when chatting with others about PR review.
You may conduct a structured review, checking each standard in turn. Doing this will be easier if you copy a template and paste it into your Github comment.
- When conducting your first or second structured review, copy template-del-1.0.md or template-mc-1.0.md. It provides several examples.
- Once you're familiar with the criteria, copy template-word-1.0.md. It's a bit shorter and quicker.
Ensure the PR has an adequate explanation.
If you were a site-builder reading the PR-log/release-notes and drilled into this PR, would you understand the description? If you were debugging a problem and traced the change back to this PR, would you understand why the change was made?
It is strongly encouraged that PR's include URLs/hyperlinks for any explanatory material (when available) -- such as a Gitlab issue, JIRA issue, StackExchange question, related PR, or Mattermost chat. However, hyperlinks are not a substitute for a description. The PR should still have a description.
PR descriptions should generally follow the pull-request template, although this could be waived if another structure is more expressive.
- WIP PRs do not need a detailed explanation until they're ready for earnest review.
- Genuine NFC PRs do not need a detailed explanation.
If a user was comfortable using the old revision, would they upgrade and assimilate naturally and unthinkingly to the new revision? If not, has there been commensurate effort to provide a fair transition-path and communication?
Some changes require adding or updating documentation. Consider the impact of this change on users, system administrators, and developers. Do they need additional instructions in order to reap the benefits of this change? If so, update documentation as necessary by making a corresponding PR on one of the guides.
Use the code somehow. You don’t need to attack every imaginable scenario in every PR, but you should do something to try it out. Be proportionate.
- Would the patch materially change the contract (signature/pre-condition/post-condition) for APIv3, a hook, a PHP function, a PHP class, a JS widget, or a CSS class?
- Would you consider the changed element to be an officially supported contract? A de-facto important contract? An obscure internal detail?
- How might the change affect other parts of
civicrm-core? extensions? third-party integrations?
- If it's hard to answer, look for inspiration:
- If there is a foreseeable problem:
- Is there a simple alternative?
- Has there been commensurate effort to communicate change and provide a fair transition path?
Is it understandable? Does it follow common conventions? Does it fit in context? If it changes a difficult section of code -- does it tend to make that section better or worse?
Many changes should introduce some kind of automated test or protective measure to ensure maintainability. However, there can be tricky cost/benefit issues, and the author and reviewer must exercise balanced judgment.
If the automated tests come back with any failures, look into why. Hopefully, Jenkins provides an itemized list of failures. If not, dig further into the "Console" output for PHP-fatals or build-failures.
If the PR adds a new top-level file, new top-level folder, or novel file-type, consider whether "distmaker" will properly convey the file in
If the PR removes a dangerous file, then common package handling may not be enough to remove the file. (This is particularly for Joomla users, but also true for with
manual file management on other platforms.) Consider updating
If the PR changes the permissions model (by adding, removing, or repurposing a permission), are we sure that demo/test builds and existing installations will continue to work as expected?
If the PR passes data between different tiers (such as SQL/PHP/HTML/CLI), is this data escaped and validated correctly? Or would it be vulnerable to SQL-injections, cross-site scripting, or similar?
If the PR adds or removes a setting, will existing deployments or build-scripts which reference the setting continue to work as expected?
Some PRs change the database by modifying schema or inserting new data. Ensure new installations and upgraded installations will end up with consistent schema. (Extra: If it's a backport, take extra care to consider all upgrade paths.)
If the upgrade adds a column or index, use a helper function to insert it. (This makes the upgrade more robust against alternative upgrade paths, such as backports/cherry-picks.)
If the upgrade needs to populate or recompute data on a large table (such as
civicrm_mailing_event_queue), the upgrade screen could timeout. Consider applying the updates in batches.
If the upgrade inserts new strings, should they be generated in a multilingual-friendly way?
See also: Upgrade Reference
Adding a new parameter to an existing hook may be syntactically safe, but is it semantically safe?