Key-value Logs
Key-value is a simple structured data format that stores a collection of key and value pairs. Some applications use key-value formatted logs for logging, and you can fetch values from these logs by specifying the respective key in a log pattern. This pattern improves the search-filter capabilities and parses out the values seamlessly.
Site24x7 helps parse your key-value logs. Additionally, Site24x7 supports the combination of a normal log with key-value log patterns. Get more information from these logs using Site24x7 AppLogs.
Getting started
- Log in to your Site24x7 account.
- Download and install the Site24x7 Server monitoring agent (Windows | Linux).
- Go to Admin > AppLogs > Log Types > Add Log Type.
You can refer to this Adding a Log Type help document for more information.
Key-value log pattern
Sample log
date="2014-06-18 11:57:46,719" at=info method=POST path="/en/admin/post/1/edit?k0=v0&k1=v1" host=helloworld-symfony.herokuapp.com request_id=e8843b25-3587-4229-a430-c93360a0e89f fwd="121.24.53.11" dyno=web.1 connect=1ms service=243ms status=302 bytes=559 protocol=https
Log pattern
The log pattern should start with the keyword keyvalue and contain a minimum of one date field. The following is the sample pattern defined by Site24x7 to parse the above key-value logs:
keyvalue $date:date:yyyy-MM-dd HH:mm:ss,SSS$ $at$ $method$ $path$ $host$ $request_id as requestid$ $fwd$ $dyno$ $connect$ $service$ $status:number$ $bytes:number$ $protocol$
If you wish to change the field name, say from request_id to requestid, use the as keyword. Also, if you want to exclude a field, you can remove it from the log pattern.
This log is separated into fields, each of which will take its respective value, and is then uploaded to Site24x7.
| Field name | Field value |
| date | 2014-06-18 11:57:46,719 |
| at | info |
| method | POST |
| path | /en/admin/post/1/edit?k0=v0&k1=v1 |
| host | helloworld-symfony.herokuapp.com |
| requestid | e8843b25-3587-4229-a430-c93360a0e89f |
| fwd | 121.24.53.11 |
| dyno | web.1 |
| connect | 1ms |
| service | 243ms |
| status | 302 |
| bytes | 559 |
| protocol | https |
Combination of normal logs with key-value pattern logs
Sample log
2012-02-07T09:43:06.123456+00:00 heroku[router]: at=info method=GET path="/stylesheets/dev-center/library.css" host=devcenter.heroku.com fwd="204.204.204.204" dyno=web.5 connect=1ms service=18ms status=200 bytes=13
Log pattern
The following log pattern contains a combination of normal and key-value logs:
$DateTime:date$ heroku[router]: $Message:keyvalue:$at$$method$ $path$ $host$ $fwd$ $dyno$ $connect$ $service$ $status:number$ $bytes:number$$
DateTime and heroku are the field names in a normal log pattern, whereas the field names followed by the Message:keyvalue keyword follow the key-value log pattern.
| Field name | Field value |
| DateTime | 2012-02-07T09:43:06.123456+00:00 |
| at | info |
| method | GET |
| path | /stylesheets/dev-center/library.css |
| host | devcenter.heroku.com |
| fwd | 204.204.204.204 |
| dyno | web.5 |
| connect | 1ms |
| service | 18ms |
| status | 200 |
| bytes | 13 |
