Linux

Access Logs and Error Logs (LOGGING BASICS) in apache

1. Locate where logged data is stored

This simple task is made harder by the fact that Apache is cross-platform software that runs on nearly every server-based operating system.

While you might expect that Apache log files will be stored in different file system locations when running on a Mac or Windows PC, you should note that different Linux distros have their own ideas where to store log data.

Apache also stores data in a number of different logging formats.

The format your Apache installation uses is defined in the Apache configuration file.

  • By default, Apache logs data uses the Common Log Format – the Common Log Format defines a label/nickname and associates it.
  • Another popular format is the Combined Log Format – this format is based on the Common Log Format, and combines it with the header of the HTTP access request.
  • Apache also lets you store log data in many common data formats, such as XML and JSON

2. EXTRACTING DATA FROM APACHE ERROR LOGS AND APACHE ACCESS LOGS

Before exploring how to use log files, we need to discuss the best way to extract data from them.

Apache access and error logs can give us valuable insights into the usage and functioning of an Apache-based website or application.

However, the sheer amount of data that is collected can be overwhelming, and finding important indicators can be a daunting process.

The obvious way to read log file data is using a basic text editor, but many editors may have a hard time dealing with large log files.

  • A common approach to reading log files, especially on Linux/Unix based systems, is to take advantage of the operating system’s command line tools, such as GREP and AWK.
  • Another approach is to use scripting languages, such as BASH or Powershell to parse log files.

3. Where is the Apache error log?

Depending on your Linux distribution, Apache log file locations may be available in different folders. One point that influences the location of the logs is the type of installation done from Apache. If you did a default installation, follow the log storage location(s).

RHEL / Red Hat / CentOS / Fedora
/var/log/httpd/error_log

Debian / Ubuntu
/var/log/apache2/error.log

FreeBSD
/var/log/httpd-error.log

To find the exact Apache error log file location(s), you can use the grep command:

$ grep ErrorLog /usr/local/etc/apache22/httpd.conf
$ grep ErrorLog /etc/apache2/apache2.conf
$ grep ErrorLog /etc/httpd/conf/httpd.conf

Sample output:

# ErrorLog: The location of the error log file.
# If you do not specify an ErrorLog directive within a
ErrorLog “/var/log/httpd-error.log”

In order to succeed in log reading, you need to know the structure of the log, and where to find it. In the case of Apache running on Linux servers, there are default paths.

The default format of the Apache error log is defined by the ErrorLogFormat directive. It takes this form:

ErrorLogFormat “[%t] [%l] [client %a] %M”
%t is time relative to the server
%l is the level of the message (error, notice, fatal, etc.)
%a is the client IP making the request (if it exists)
%M is the actual error message

An example:

[Sun Jul 09 03:25:02 2017] [notice] Apache/2.2.32 (Unix) DAV/2 configured -- resuming normal operations
[Sun Jul 09 04:06:13 2017] [error] [client 1.2.3.4] File does not exist: /var/www/html/robots.txt
[Mon Jul 10 20:24:52 2017] [error] (111)Connection refused: proxy: HTTP: attempt to connect to 127.0.0.1:8484 (localhost) failed

4. View Apache error logs

When you have a tool for displaying the data stored in a log, you can quickly find the answers you need. To monitor Apache logs, the easy-to-use Sumo Logic Apache App service is a great choice. You can quickly install the application and configure it to receive the data from your server, and display needed information.

The data that can be monitored are:

  • Bytes Delivered
  • HTTP Errors
  • Page Hits
  • Response Time
  • Web Server

This data allows you to work safely, and analyze all traffic your application receives.

5. Apache Error Log Analysis

Apache logs contain valuable operational data. Apache error log analysis from Sumo Logic can provide insight and alerts to server errors, request processing, and other diagnostic issues. Analyzing Apache error logs has never been easier, and with Sumo Logic’s built-in error log parser, you can quickly visualize your logs without writing any regex.