How-Tos

These guides will help you manage logs with practical step-by-step procedures.

Export Logs to a Log Management System
Use the logging configuration to send Usercube's logs to a log management system.

Monitoring

Usercube uses Serilog, a highly customizable logging tool, to provide monitoring capabilities.

Introduction

Serilog configuration is written to both Agent's and Server's appsettings sets. The relevant top-level section is Serilog.

A full description of Serilog's configuration capabilities is available in Serilog's official documentation.

Usercube-specific configuration is detailed here.

Log Level and Namespaces

Priority

Logs can be filtered according to a log level.

A priority order between the log levels is established.

From low priority to high priority, available log levels are:

Verbose
Debug
Information
Warning
Error
Fatal

Every log message is associated with a log level and a user-defined namespace. Usercube provides the Usercube namespace, associated with logs relevant to the user.

MinimumLevel

The MinimumLevel section sets the lowest priority log level that will be displayed. Every log message associated with a log level of priority strictly lower than the minimum level is ignored.

MinimumLevel value can either be a log level or an object with the following attributes and subsections:

Default sets the minimum log level.
Override allows the user to set a different minimum log level for logs from a specific namespace (see Custom namespaces).

Within Usercube, the following example is a good practice: default logs with a priority lower than Error are filtered out, except for log messages from the Usercube namespace.

appsettings.json
{
  ...
  "Serilog": {
    ...
    "MinimumLevel": {
      "Default": "Error",
      "Override": {
        "Usercube": "Information"
      }
    }
  }
}

Custom namespaces

Here is a table giving some namespace that you could add in the Override section, in order to monitor the associated module.

Module	Namespace
Usercube	Usercube
Scheduler (server side)	Usercube.Jobs.Scheduler.Server
Scheduler (agent side)	Usercube.Jobs.Scheduler

Log Properties

Each log has a specific set of log properties, defined using the context of the server when generating the log (see Formatting).

It is possible to modify the format message of the log displayed by overriding the outputTemplate of the logs:

appsettings.json
{
  ...
  "Serilog": {
    "MinimumLevel": {
      "Default": "Verbose",
    },
    "WriteTo": [
      {
        "Name": "Console",
        "Args": {
          "outputTemplate": "[{Timestamp:HH:mm:ss} {Level:u3}] ClientId:{ClientId} {Message:lj}{NewLine}{Exception}"
        }
      }
    ]
  }
}

Among all default properties, Usercube adds the ClientId log property which can be displayed when using the previous outputTemplate format.

Filters

In addition to the Microsoft log levels, Serilog provides a Filters feature to build more advanced filter queries on log messages.

Sinks

Serilog allows the user to route log messages to a variety of logging destinations. Every destination is referred to as a sink. Sinks allows logs to be routed to destination such as standard consoles, files and logging services.

Usercube's supported sinks are:

Serilog.Sinks.ApplicationInsights;
Serilog.Sinks.Async;
Serilog.Sinks.Console to write to the console;
Serilog.Sinks.Datadog.Logs;
Serilog.Sinks.File to write to a file;
Serilog.Sinks.Map;
Serilog.Sinks.Network to write to another network;

For example, this sink can be used when producing a JSON output for QRadar.
Serilog.Sinks.PeriodicBatching;
Serilog.Sinks.Splunk.Durable to send logs to Splunk;
Serilog.Sinks.Syslog.

For example, this sink can be used when producing an RFC3164 or RFC5424 output for QRadar.

The log messages can be routed to several logging destinations simultaneously. These destinations are described in the WriteTo attribute.

appsettings.json
{
  ...
"Serilog": {
    "Using": [
      "Serilog.Sinks.Network"
    ],
    "MinimumLevel": {
      "Default": "Error",
      "Override": {
          "Usercube": "Information"
      }
    },
    "WriteTo": [
      {
        "Name": "Destination1",
        "Args": {
          "uri": "192.168.13.110",
          "port": "514",
          "textFormatter": "Serilog.Formatting.Compact.CompactJsonFormatter, Serilog.Formatting.Compact"
        }
      },
      {
        "Name": "Destination2",
        "Args": {
          "uri": "192.168.13.227",
          "port": "514",
          "textFormatter": "Serilog.Formatting.Compact.CompactJsonFormatter, Serilog.Formatting.Compact"
        }
      }
    ],
    "Filter": [
      {
        "Name": "ByIncludingOnly",
        "Args": { "expression": "StartsWith(SourceContext, 'Usercube') and EventId.Id >= 500" }
      }
    ]
  }
}

There can only be one Filter attribute associated with a WriteTo attribute. Therefore, the filter defined in the Filter attribute is applied to all the destinations contained in the WriteTo attribute. To filter only one destination at a time, sub-loggers can be used.

appsettings.json
{
  ...
"Serilog": {
    "Using": [
      "Serilog.Sinks.Network"
    ],
    "MinimumLevel": {
      "Default": "Error",
      "Override": {
          "Usercube": "Information"
      }
    },
    "WriteTo": [
      {
        "Name": "Logger1",
        "Args": {
          "configureLogger": {
            "MinimumLevel": {
                "Default": "Information"
            },
            "WriteTo": [
              {
                "Name": "Destination1",
                "Args": {
                  "uri": "192.168.13.127",
                  "port": "514",
                  "textFormatter": "Serilog.Formatting.Compact.CompactJsonFormatter, Serilog.Formatting.Compact"
                }
              }
            ],
            "Filter": [
              {
                "Name": "ByIncludingOnly",
                "Args": { "expression": "StartsWith(SourceContext, 'Usercube') and EventId.Id >= 500" }
              }
            ]
          }
        }
      },
      {
        "Name": "Logger2",
        "Args": {
          "configureLogger": {
            "MinimumLevel": {
                "Default": "Information"
            },
            "WriteTo": [
              {
                "Name": "Destination2",
                "Args": {
                  "uri": "192.168.13.100",
                  "port": "514",
                  "textFormatter": "Serilog.Formatting.Compact.CompactJsonFormatter, Serilog.Formatting.Compact"
                }
              },
              {
                "Name": "Destination3",
                "Args": {
                  "uri": "192.168.13.408",
                  "port": "514",
                  "textFormatter": "Serilog.Formatting.Compact.CompactJsonFormatter, Serilog.Formatting.Compact"
                }
              }
            ],
            "Filter": [
              {
                "Name": "ByIncludingOnly",
                "Args": { "expression": "StartsWith(SourceContext, 'Test') and EventId.Id >= 800" }
              }
            ]
          }
        }
      }
    ]
  }
}

In the example above, the filter defined in Logger1 will only apply to Destination1, and the filter defined in Logger2 will only apply to Destination2 and Destination3.

When using Serilog.Sinks.File, the setting shared should be set to true in the Args section to enable Usercube's Monitoring screen functionality.

As this shared setting allows several systems to interact with the log file simultaneously, so we can have both Serilog writing to the log file and Usercube reading it to display its content on the Monitoring screen.

{
  ...
  "Serilog": {
    "WriteTo": [
        {
          "Name": "File",
          "Args": {
            "path": "../Temp/Server/identitymanager-log.txt",
            "shared": true,
          }
        }
    ]
  }
}

QRadar

QRadar is a supported destination for Usercube's logs.

To learn how to send Usercube's logs to your QRadar system, see dedicated How To page .

Three output formats are available for QRadar-routed logs:

JSON
RFC3164
RFC5424

JSON output

JSON output uses Serilog.Sinks.Network sink.

The following configures a QRadar JSON output for a QRadar server located at 192.168.13.110.

appsettings.json
{
  ...
"Serilog": {
    "Using": [
      "Serilog.Sinks.Network"
    ],
    "MinimumLevel": {
      "Default": "Error",
      "Override": {
          "Usercube": "Information"
      }
    },
    "WriteTo": [
      {
        "Name": "Logger",
        "Args": {
          "configureLogger": {
            "MinimumLevel": {
                "Default": "Information"
            },
            "WriteTo": [
              {
                "Name": "UDPSink",
                "Args": {
                  "uri": "192.168.13.110",
                  "port": "514",
                  "textFormatter": "Serilog.Formatting.Compact.CompactJsonFormatter, Serilog.Formatting.Compact"
                }
              }
            ],
            "Filter": [
              {
                "Name": "ByIncludingOnly",
                "Args": { "expression": "StartsWith(SourceContext, 'Usercube') and EventId.Id >= 500" }
              }
            ]
          }
        }
      }
    ]
  }
}

RFC3164 or RFC5424 output

Using Serilog.Sinks.SyslogMessagesSink, the Serilog.writeTo.configureLogger.Args.format attribute is set to RFC3164 or RFC5424.

The following configures a QRadar RFC5424 output for a QRadar server located at 192.168.13.110.

appsettings.json
{
  ...
"Serilog": {
    "Using": [
      "Serilog.Sinks.Syslog"
    ],
    "MinimumLevel": {
      "Default": "Error",
      "Override": {
          "Usercube": "Information"
      }
    },
    "WriteTo": [
      {
        "Name": "Logger",
        "Args": {
          "configureLogger": {
            "MinimumLevel": {
                "Default": "Information"
            },
            "WriteTo": [
              {
                "Name": "UdpSyslog",
                "Args": {
                  "host": "192.168.13.110",
                  "port": "514",
                  "appName": "Usercube",
                  "format": "RFC5424",
                  "facility": "Local0",
                  "secureProtocols": "SecureProtocols.None",
                  "outputTemplate": "[{Timestamp:HH:mm:ss} {Level:u3}] {Message:lj}

Application Insights

Usercube supports the Application Insights integration. It means that you can monitor the lifecycle of the application through a dedicated interface, which can be useful to measure performance, observe how the application is used or detect performance anomalies.

Configuration

Both the server and the agent support the Application Insights integration. To set it up, you need to create your own Application Insights instance (see Create New Resource). Once done, you should have an instrumentation key. To plug the server or the agent into the Application Insights instance, you simply have to set the key at the root of the appsettings file:

appsettings.json
{
  ...
  "ApplicationInsights": {
    "InstrumentationKey": "YOUR-INSTRUMENTATION-KEY"
  }
}

This configuration will automatically add a Serilog.Sinks.ApplicationInsights to the Serilog configuration. Thus, declaring explicitly an ApplicationInsights sink in the Serilog configuration is useless. The ApplicationInsights section does not only affect the logging system, but also sends metrics periodically such as the percentage of CPU usage.

Logs Monitoring via User Interface

Usercube offers the ability to download the application logs directly through the User Interface (UI) via the Monitoring screen in the Administration section on the Dashboard.

SaaS installations support this feature automatically while on-premises installations support this in two ways. The first one is to leverage the path to the logs from the Serilog configuration when writing application logs into a single file. See the example below. The second option is described in the following subsection.

appsettings.json
{
  ...
  "Serilog": {
    "WriteTo": [
        {
          "Name": "File",
          "Args": {
            "path": "../Temp/Server/identitymanager-log.txt",
            "shared": true,
          }
        }
    ]
  }
}

`LogsPath`

if you store Usercube logs thanks to an external mechanism (the web server, etc�), then you have to use the second option in order to enable this feature which is via an ad hoc parameter at the root of the appsettings called LogsPath indicating the path where the application logs are located:

appsettings.json
{
  ...
  "Serilog": {
    "WriteTo": [ "Console" ],
    },
  "LogsPath": "C:/inetpub/logs/LogFiles"
}

If logs are all stored in one file, provide the path to the file. If they are stored in multiple separate files within a directory, provide the path to the directory and Usercube will handle providing the most recent logs.

Default Configuration

appsettings.json
{
  ...
"Serilog": {
  "WriteTo": [ "Console" ],
  "Using": [ "Serilog.Sinks.File" ],
  "MinimumLevel": "Error",
  "WriteTo": [
      {
        "Name": "File",
        "Args": {
          "path": "../Temp/Server/identitymanager-log.txt",
          "shared": true
        }
      }
  ]
}
}

Configuration Examples

Write log messages

This example configures Serilog to write log messages to the ../Temp/Server/identitymanager-log.txt file.

appsettings.json
{
  ...
"Serilog": {
  "WriteTo": [ "Console" ],
  "Using": [ "Serilog.Sinks.File" ],
  "MinimumLevel": "Error",
  "WriteTo": [
      {
        "Name": "File",
        "Args": {
          "path": "../Temp/Server/identitymanager-log.txt",
          "shared": true
        }
      }
  ]
}
}

Reduce logging process overhead

This example shows how to reduce the overhead of the logging process for Usercube's main thread by delegating work to a background thread, using the Async__Sink.

appsettings.json
{
  ...
"Serilog": {
    "MinimumLevel": {
      "Default": "Error",
      "Override": {
        "Usercube": "Debug"
      }
    },
    "WriteTo": [
      {
        "Name": "Async",
        "Args": {
          "configure": [
            {
              "Name": "File",
              "Args": {
                "path": "C:/Projects/LogTest/identitymanager-test.txt",
                "shared: true,
                "buffered": "true"
              }
            }
          ]
        }
      },
      {
        "Name": "Console"
      }
    ]
  }
}

References: Logs

Definition

This section provides descriptions for logs which are meant to be sent to other systems like SIEMs, for example QRadar.

The description will use this template for each log:

EventId id: int

EventId name: string

LogLevel: Trace||Verbose||Debug||Information||Warning||Error||Critical

Arguments:

argument1 (string): description1 (string)
argument2 (string): description2 (string)
argument3 (string): description3 (string)

The EventId id must be unique so we could use it to filter the logs we send, see QRadar's example

500

EventId id: 500

EventId name: Workflow.StartWorkflowInstance

LogLevel: Information

Arguments:

WorkflowId: Request number, which includes the workflow instance's id
Transition: Activity template name
Perfomer: Usercube's login or id of the performer
WorkflowIdentifier: Workflow's identifier
Subject: Action performed, with the person's name in modifying permission case

501

EventId id: 501

EventId name: Workflow.ResumeWorkflowInstance

LogLevel: Information

Arguments:

WorkflowId: Request number, which includes the workflow instance's id
Transition: Activity template name
Perfomer: Usercube's login or id of the performer
WorkflowIdentifier: Workflow's identifier
Subject: Action performed, with the person's name in modifying permission case

502

EventId id: 502

EventId name: SelectEntityByIdQueryHandler.Handle

LogLevel: Information

Arguments:

Perfomer: Usercube's login or id of the performer
Subject: Usercube's id of the readed resource
EntityType: Usercube's type of the readed resource

503

EventId id: 503

EventId name: SelectEntityByIdQueryHandler.Handle

LogLevel: Error

Arguments:

Perfomer: Usercube's login or id of the performer
Subject: Usercube's id of the readed resource
EntityType: Usercube's type of the readed resource
ExceptionMessage: Exception's message

Export Logs to a Log Management System​

Monitoring

Introduction​

Log Level and Namespaces​

Priority​

MinimumLevel​

Custom namespaces​

Log Properties​

Filters​

Sinks​

QRadar​

JSON output​

RFC3164 or RFC5424 output​

Application Insights​

Configuration​

Logs Monitoring via User Interface​

LogsPath​

Default Configuration​

Configuration Examples​

Write log messages​

Reduce logging process overhead​

References: Logs

Definition​

500​

501​

502​

503​

Export Logs to a Log Management System

Introduction

Log Level and Namespaces

Priority

MinimumLevel

Custom namespaces

Log Properties

Filters

Sinks

QRadar

JSON output

RFC3164 or RFC5424 output

Application Insights

Configuration

Logs Monitoring via User Interface

`LogsPath`

Default Configuration

Configuration Examples

Write log messages

Reduce logging process overhead

Definition

500

501

502

503