[Development] (long) thoughts on categorized logging (qLog)

[Lincoln Ramsay]

Hi all,

There seems to be some confusion over what qLog is and is not. I suspect 
that the direction was perhaps mis-aligned slightly. Please allow me to 
try to summarize things. This is more of a vision thing and in writing 
this, parts of the implementation have been identified that should be 
updated. Writing this led to much discussion here, this represents 
something we are happy with and I'm hoping it matches the expectations 
of others too. Note that some different names are being proposed here, 
the changes to functionality suggested the names but as before, they're 
not set in stone yet.

First a mission statement:

The categorized logging feature is intended as a replacement for qDebug 
and qWarning. With logging statements in an app/library, a developer can 
turn on the statements they care about and turn off the ones they don't. 
They can do all this without re-compiling the app and in some cases, 
without restarting the app. The categorized logging feature is intended 
to have as small an impact as possible until it has been explicitly enabled.


The categorized logging feature is made up of several parts.


PART 1: The logging statements.

To keep the code compact (and for performance reasons), categories are 
identified locally by a short (locally-unique) token. The short tokens 
are mapped onto longer, (globally-unique) identifiers. Categories must 
be declared before use.

QT_LOG_CATEGORY(PLUGIN, "com.nokia.qt.plugins")

For a library, I'd expect to see a header that declares the categories 
used by that library but it's also possible to just drop the macro in 
the top of .cpp files as required. (impl detail: might need a "declare" 
and a "implement" macro for libs to avoid duplicate symbols)

The actual logging statements use the category token.

qCDebug(PLUGIN) << "Scanning for plugins in" << paths;
qCWarning(PLUGIN) << "Could not load plugin" << loader->errorString();

As can be seen, we're differentiating between debug (defualt off) and 
warning (default on) statements. To identify these when turning 
categories on/off there is a .debug or .warning suffix added to the 
identifier (more on this later).

It's important to note that logging statements should not have side 
effects because they may not be called.


PART 2: Compatibility with qDebug/qWarning

qDebug() and qWarning() will be re-implemented as categorized logging 
statements. Both will use the same, hard-coded category.

qDebug() << "debug";
qWarning() << "warning";

Is almost equivalent to:

QT_LOG_CATEGORY(LEGACY, "legacy")
qCDebug(LEGACY) << "debug";
qCWarning(LEGACY) << "warning";

I say "almost equivalent" because qDebug() defaults on (while 
categorized debugs default off).

The older qDebug("debug") and qWarning("warning") syntax will also work 
and will be treated the same way.

For display purposes, qFatal will also have a category but it won't be 
influenced by the logging configuration.


PART 3: Turning statements on/off.

As mentioned, debug statements (except for qDebug) default off and 
warning statements default on. Logging statements can be enabled or 
disabled using rules. Rules specify a category identifier, optionally 
using a global-style * to match multiple categories.

eg.
# Turn on Qt plugin debug
com.nokia.qt.plugins.debug = true
# Turn off network warnings
com.nokia.qt.network.warning = false
# Turn on all webkit debug
org.webkit.*.debug = true
# Turn off all legacy qDebug/qWarning
legacy.* = false

For apps that want to provide a UI for logging and for future expansion 
possibilities (eg. add-ons that allow turning logging on/off via some 
novel means), there will be a C++ API to set the rules.

qSetCategoryRules(const QByteArray &rules);

However, since there are likley to be many apps that don't care about 
logging (or may not expose a way to turn on logging in a third party 
library) there will also be a text-based config file that the user can 
create. Where should this file be located? Either the app sets the 
location of this file by calling qSetLoggingConfigFile() or the user 
sets the QT_LOGGING_CONFIG environment variable (which overrides the 
app's default). If the config file exists, the C++ API has no effect. 
QFileWatcher will be used to monitor the config file so that categories 
can be turned on/off while the program is running by updating the file.


PART 4: Formatting.

There's a %{category} entry that can appear in the format string. This 
will be in the default format string.


PART 5: Saving logs to a file.

As a special case, the user can direct enabled messages to a file. This 
is done from the config file.

# write output to a file (in the user's home directory)
logging.output.file = file.txt
# write output to a specific file
logging.output.file = /path/to/file.txt
# write output.file to a specific file on Windows
logging.output.file = c:\log.txt

# this is the default (output goes to message handler or file, not both)
logging.output.both = false
# write to both the file and the message handler
logging.output.both = true

# for messages being written into the file, overwrite the formatting string
logging.output.format = "%{category}: %{message}"

Ideally, this would exist in an add-on. In this case we feel it is 
justified to build this into the library due to the difficulty of 
collecting logs from apps on different platforms.



-- 
Lincoln Ramsay - Senior Software Engineer
Qt Development Frameworks, Nokia - http://qt.nokia.com/