Need the ability to track web traffic for a specific hours worth of logs.
Using GoAccess to parse Apache logs that are rotated into hour chunks.
Install GoAccess to parse webserver logs. Use Ansible Playbooks for the installation, configuration and for re-usabilty of GoAccess on other servers. Keep it simple by rotating logs on the hour.
- GoAccess - real-time web log analyzer - allinurl/goaccess
- Ansible - Ansible Docs
- Apache rotatelogs - Hourly Rotated WebServer Logs
- Digital Ocean - My referral link, thanks!
- TugBoat - CLI for DO - pearkes/tugboat
Ansible Playbook - Install GoAccess
Create the GoAccess Playbook. This installs GoAccess using Apt on Debian.
Run the playbook with Ansible.
sandor@theargo ansible-playbook ~/playbooks/goaccess/goaccess.yml -vvvv
Now that GoAccess is installed on the server we can login and parse the webserver logs.
sandor@argo:/var/log/apache2$ goaccess -f apache2.log
GoAccess now parses the log and outputs the results in a configurable format. You can also export the results into various file formats.
Here is a screenshot that shows how many times an IP has hit the webserver.
To keep it simple install Apache rotatelog and set the logs to rollover on the hour. Then GoAccess can parse the logs by the hour.
Apache rotatelog setting for hourly logs:
CustomLog "|bin/rotatelogs -l /var/log/apache2.%Y-%m-%d-%H 3600" common
Parsing a specific hours logs with GoAccess
Example - Logs for 8am April, 2 2014.
sandor@argo> /var/log/apache2$ goaccess -f apache2.2014-04-02-08