Skip to content

Log Reader

Search CiviCRM's debug log — the one that goes in [civicrm.log], usually files/civicrm/ConfigAndLog/.

Find it at Administer → Administration Console → Debug Log.

The extension is licensed under AGPL-3.0.

Features

  • Search the full text of every entry, stack traces included, so you can find the class or file named in a backtrace and not only in the message line. The log viewers that search only the first line of an entry will not find these.
  • Filter by date range and by severity.
  • Open rotated log files as well as the one currently being written to. Each is listed with its size and the date of its first entry, so you can tell which file covers the period you care about.
  • Refresh without losing your place. A log grows while you are reading it; the Refresh button re-runs the current search in place, keeping the filters and showing when the list was last loaded. Set it to check every few seconds and the button reports how much has been written since, or updates the list by itself. Checking costs a stat of each log file, not a search of it.
  • Opens immediately on very large logs. Log files are read as a stream and are never loaded into memory whole. Showing the newest entries reads only the end of the file: on a 269MB log holding 2.4 million entries, the first page costs a couple of block reads. Filtering to a date range bisects the file to find that period rather than reading up to it, so a search bounded to one day of that log costs the same whether the day is at the start or the end.

Clicking an entry's date opens it in full, including any stack trace. Previous and Next then step through the entries the filters matched, skipping everything between, so a search that finds a handful of entries in a million is something you can read through. Previous is always older and Next always newer, whichever way the table is sorted, and a button is shown only when there is actually a matching entry that way. With no filters applied they step through the file, as before.

Installation

Learn more about installing CiviCRM extensions in the CiviCRM Sysadmin Guide.

Requires CiviCRM 6.17 or later, with SearchKit and Afform enabled.

The DebugLogEntry API

Log entries are exposed as an APIv4 entity, DebugLogEntry, with the fields id, file, timestamp, severity, message, body and size.

The entity is searchable, so you can build your own SearchKit searches, displays and dashboards over your logs — the bundled viewer is itself just a saved search and a table display.

// Errors mentioning a class, anywhere in the entry including the stack trace
civicrm_api4('DebugLogEntry', 'get', [
  'where' => [
    ['severity', 'IN', ['error', 'critical']],
    ['body', 'CONTAINS', 'CRM_Core_Payment'],
  ],
  'limit' => 25,
]);

body holds the whole entry and is what a text search matches against; message holds only the first line, which is what the table displays.

get also takes a startAfter parameter, an entry id the scan resumes just past rather than starting at the end of the file. It follows orderBy, so the same where clause run once descending and once ascending gives the entries either side of that one — which is how the entry viewer's Previous and Next find the adjacent matching entry without re-reading everything before it.

Limitations

Because the records are files rather than database tables:

  • the entity cannot be joined to other entities, and does not support groupBy or having;
  • sorting by timestamp (the default, newest first) is answered by reading the file from the appropriate end, but sorting by any other column has to gather rows first and so considers only the most recent 10,000 matches. This is recorded in the log when it happens.
  • the entries either side of the one you are viewing are found by scanning out from it, so under a text search with no date range opening an entry can read the whole file — the same cost as the search that found it. A date range bounds it.

Totals for the pager have to consider every entry in the period, so they are cached against each file's size and modification time. A growing log invalidates its own cached totals.

Notes

Parsing relies on the timestamp format CiviCRM's own logger writes (Y-m-d H:i:sO). A site that has changed that format, or writes its log in another language, may not parse correctly.

Reading a log never rotates it. CiviCRM's own CRM_Core_Error::createDebugLogger() renames the live log once it exceeds CIVICRM_LOG_ROTATESIZE or the month rolls over; this extension determines the current file from the directory listing instead, so opening the page has no side effects.

Credits

Inspired by ca.civicrm.logviewer by Alan Dixon of Blackfly Solutions, which established the idea of reading CiviCRM's debug log from within CiviCRM. The implementation here is new: entries are read as a stream and exposed as an APIv4 entity so that SearchKit provides the interface.