Compare commits
22 Commits
Author | SHA1 | Date |
---|---|---|
Mark Nellemann | 3ef961e44b | |
Mark Nellemann | bd5e2634d5 | |
Mark Nellemann | 1acdd6a93d | |
Mark Nellemann | d39837861f | |
Mark Nellemann | 422f1fbb71 | |
Mark Nellemann | d3589faf9e | |
Mark Nellemann | c47f682c34 | |
Mark Nellemann | b291f87693 | |
Mark Nellemann | 3e7ba2f46e | |
Mark Nellemann | a0ad98bf52 | |
Mark Nellemann | 9fb810c9d2 | |
Mark Nellemann | 8895b2b110 | |
Mark Nellemann | 99466b0037 | |
Mark Nellemann | 2284b5eecb | |
Mark Nellemann | cd6e15584a | |
Mark Nellemann | 5104bd0750 | |
Mark Nellemann | 4cc11b0587 | |
Mark Nellemann | 4cb04bd687 | |
Mark Nellemann | 4a05014dbc | |
Mark Nellemann | 2b52550f87 | |
Mark Nellemann | 3e21ea395e | |
Mark Nellemann | cf377adecd |
|
@ -0,0 +1,23 @@
|
||||||
|
---
|
||||||
|
kind: pipeline
|
||||||
|
name: default
|
||||||
|
type: docker
|
||||||
|
|
||||||
|
steps:
|
||||||
|
- name: build
|
||||||
|
image: eclipse-temurin:8-jdk
|
||||||
|
commands:
|
||||||
|
- ./gradlew build
|
||||||
|
- name: publish
|
||||||
|
image: eclipse-temurin:8-jdk
|
||||||
|
environment:
|
||||||
|
AUTH_TOKEN: # Gitea access token ENV variable
|
||||||
|
from_secret: auth # Name of DroneCI secret exposed above
|
||||||
|
commands:
|
||||||
|
- ./gradlew packages
|
||||||
|
- for file in build/libs/*-all.jar ; do curl --user "${DRONE_REPO_OWNER}:$${AUTH_TOKEN}" --upload-file "$${file}" "https://git.data.coop/api/packages/${DRONE_REPO_OWNER}/generic/${DRONE_REPO_NAME}/${DRONE_TAG}/$(basename $file)" ; done
|
||||||
|
- for file in build/distributions/*.deb ; do curl --user "${DRONE_REPO_OWNER}:$${AUTH_TOKEN}" --upload-file "$${file}" "https://git.data.coop/api/packages/${DRONE_REPO_OWNER}/generic/${DRONE_REPO_NAME}/${DRONE_TAG}/$(basename $file)" ; done
|
||||||
|
- for file in build/distributions/*.rpm ; do curl --user "${DRONE_REPO_OWNER}:$${AUTH_TOKEN}" --upload-file "$${file}" "https://git.data.coop/api/packages/${DRONE_REPO_OWNER}/generic/${DRONE_REPO_NAME}/${DRONE_TAG}/$(basename $file)" ; done
|
||||||
|
when:
|
||||||
|
event:
|
||||||
|
- tag
|
98
README.md
98
README.md
|
@ -1,97 +1,3 @@
|
||||||
# Syslog Server
|
# Repository moved
|
||||||
|
|
||||||
All received messages are written to *stdout* and/or forwarded to a remote logging destination.
|
|
||||||
|
|
||||||
The syslog server is able to listen on both UDP and TCP and parses syslog messages in either RFC5424 or RFC3164 (BSD) format.
|
|
||||||
|
|
||||||
This software is free to use and is licensed under the [Apache 2.0 License](https://bitbucket.org/mnellemann/syslogd/src/master/LICENSE).
|
|
||||||
|
|
||||||
![architecture](https://bitbucket.org/mnellemann/syslogd/downloads/syslogd.svg)
|
|
||||||
|
|
||||||
The default syslog port (514) requires you to run syslogd as root / administrator.
|
|
||||||
If you do not wish to do so, you can choose any port number (with the *-p* or *--port* flag) above 1024.
|
|
||||||
|
|
||||||
Supported remote logging destinations are:
|
|
||||||
- Syslog (RFC5424 over UDP)
|
|
||||||
- Graylog (GELF over UDP)
|
|
||||||
- and Grafana Loki (HTTP over TCP).
|
|
||||||
|
|
||||||
## Usage Instructions
|
|
||||||
|
|
||||||
- Install the syslogd package (*.deb* or *.rpm*) from [downloads](https://bitbucket.org/mnellemann/syslogd/downloads/) or build from source.
|
|
||||||
- Run *bin/syslogd*, use the *-h* option for help :)
|
|
||||||
|
|
||||||
```text
|
|
||||||
Usage: syslogd [-dhV] [--[no-]ansi] [--[no-]stdout] [--[no-]tcp] [--[no-]udp]
|
|
||||||
[--rfc5424] [-g=<uri>] [-l=<url>] [-p=<num>] [-s=<uri>]
|
|
||||||
-d, --debug Enable debugging [default: 'false'].
|
|
||||||
-g, --gelf=<uri> Forward to Graylog <udp://host:port>.
|
|
||||||
-h, --help Show this help message and exit.
|
|
||||||
-l, --loki=<url> Forward to Grafana Loki <http://host:port>.
|
|
||||||
--[no-]ansi Output ANSI colors [default: true].
|
|
||||||
--[no-]stdout Output messages to stdout [default: true].
|
|
||||||
--[no-]tcp Listen on TCP [default: true].
|
|
||||||
--[no-]udp Listen on UDP [default: true].
|
|
||||||
-p, --port=<num> Listening port [default: 514].
|
|
||||||
--rfc5424 Parse RFC-5424 messages [default: RFC-3164].
|
|
||||||
-s, --syslog=<uri> Forward to Syslog <udp://host:port> (RFC-5424).
|
|
||||||
-V, --version Print version information and exit.
|
|
||||||
```
|
|
||||||
|
|
||||||
### Examples
|
|
||||||
|
|
||||||
Listening on a non-standard syslog port:
|
|
||||||
|
|
||||||
```
|
|
||||||
java -jar /path/to/syslogd-x.y.z-all.jar --port 1514
|
|
||||||
```
|
|
||||||
|
|
||||||
or, if installed as a *deb* or *rpm* package:
|
|
||||||
|
|
||||||
```
|
|
||||||
/opt/syslogd/bin/syslogd --port 1514
|
|
||||||
```
|
|
||||||
|
|
||||||
Listening on the standard syslog port (requires root privileges) and forwarding messages on to another log-system on a non-standard port.
|
|
||||||
|
|
||||||
```
|
|
||||||
java -jar /path/to/syslogd-x.y.z-all.jar --syslog udp://remotehost:514
|
|
||||||
```
|
|
||||||
|
|
||||||
Forwarding to a Graylog server in GELF format.
|
|
||||||
|
|
||||||
```
|
|
||||||
java -jar /path/to/syslogd-x.y.z-all.jar --gelf udp://remotehost:12201
|
|
||||||
```
|
|
||||||
|
|
||||||
Forwarding to a Grafana Loki server.
|
|
||||||
|
|
||||||
```
|
|
||||||
java -jar /path/to/syslogd-x.y.z-all.jar --loki http://remotehost:3100
|
|
||||||
```
|
|
||||||
|
|
||||||
If you don't want any output locally (only forwarding), you can use the ```--no-stdout``` flag.
|
|
||||||
|
|
||||||
|
|
||||||
## Notes
|
|
||||||
|
|
||||||
### IBM AIX and VIO Servers
|
|
||||||
|
|
||||||
Syslog messages from AIX (and IBM Power Virtual I/O Servers) can be troublesome with some logging solutions. These can be received with
|
|
||||||
*syslogd* and then forwarded on to your preferred logging solution.
|
|
||||||
|
|
||||||
### Forwarding to Grafana Loki
|
|
||||||
|
|
||||||
Forwarding is currently done by making HTTP connections to the Loki API, which works fine for low volume messages, but might cause issues for large volume of messages.
|
|
||||||
|
|
||||||
## Development Notes
|
|
||||||
|
|
||||||
### Test Grafana Loki
|
|
||||||
|
|
||||||
Run Loki and Grafana in local containers to test.
|
|
||||||
|
|
||||||
```shell
|
|
||||||
docker run --rm -d --name=loki -p 3100:3100 grafana/loki
|
|
||||||
docker run --rm -d --name=grafana --link loki:loki -p 3000:3000 grafana/grafana:7.1.3
|
|
||||||
```
|
|
||||||
|
|
||||||
|
Please visit [github.com/mnellemann/syslogd](https://github.com/mnellemann/syslogd)
|
|
@ -1,4 +1,4 @@
|
||||||
image: openjdk:8
|
image: eclipse-temurin:8-jdk
|
||||||
|
|
||||||
pipelines:
|
pipelines:
|
||||||
branches:
|
branches:
|
||||||
|
|
68
build.gradle
68
build.gradle
|
@ -2,9 +2,10 @@ plugins {
|
||||||
id 'java'
|
id 'java'
|
||||||
id 'groovy'
|
id 'groovy'
|
||||||
id 'application'
|
id 'application'
|
||||||
id "com.github.johnrengelman.shadow" version "7.1.2"
|
id 'jacoco'
|
||||||
id "net.nemerosa.versioning" version "2.15.1"
|
id "net.nemerosa.versioning" version "2.15.1"
|
||||||
id "nebula.ospackage" version "9.1.1"
|
id "com.netflix.nebula.ospackage" version "11.5.0"
|
||||||
|
id "com.github.johnrengelman.shadow" version "7.1.2"
|
||||||
}
|
}
|
||||||
|
|
||||||
repositories {
|
repositories {
|
||||||
|
@ -12,25 +13,64 @@ repositories {
|
||||||
}
|
}
|
||||||
|
|
||||||
dependencies {
|
dependencies {
|
||||||
annotationProcessor 'info.picocli:picocli-codegen:4.6.2'
|
annotationProcessor 'info.picocli:picocli-codegen:4.7.5'
|
||||||
implementation 'info.picocli:picocli:4.6.3'
|
implementation 'info.picocli:picocli:4.7.5'
|
||||||
implementation 'org.slf4j:slf4j-api:1.7.36'
|
implementation 'org.slf4j:slf4j-api:2.0.9'
|
||||||
implementation 'org.slf4j:slf4j-simple:1.7.36'
|
implementation 'org.slf4j:slf4j-simple:2.0.9'
|
||||||
|
implementation 'com.fasterxml.jackson.core:jackson-databind:2.15.2'
|
||||||
|
implementation 'com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.15.2'
|
||||||
|
implementation 'org.apache.commons:commons-collections4:4.4'
|
||||||
|
|
||||||
testImplementation('org.spockframework:spock-core:2.0-groovy-3.0')
|
testImplementation 'org.spockframework:spock-core:2.3-groovy-3.0'
|
||||||
testImplementation 'org.slf4j:slf4j-api:1.7.36'
|
|
||||||
testRuntimeOnly('org.slf4j:slf4j-simple:1.7.36')
|
|
||||||
}
|
}
|
||||||
|
|
||||||
application {
|
application {
|
||||||
getMainClass().set('biz.nellemann.syslogd.Application')
|
getMainClass().set('biz.nellemann.syslogd.Application')
|
||||||
}
|
}
|
||||||
|
|
||||||
|
java {
|
||||||
|
sourceCompatibility = 1.8
|
||||||
|
targetCompatibility = 1.8
|
||||||
|
}
|
||||||
|
|
||||||
test {
|
test {
|
||||||
useJUnitPlatform()
|
useJUnitPlatform()
|
||||||
}
|
}
|
||||||
|
|
||||||
apply plugin: 'nebula.ospackage'
|
jacoco {
|
||||||
|
toolVersion = "0.8.10"
|
||||||
|
}
|
||||||
|
|
||||||
|
jacocoTestReport {
|
||||||
|
group = "verification"
|
||||||
|
reports {
|
||||||
|
xml.required = false
|
||||||
|
csv.required = false
|
||||||
|
html.destination file("${buildDir}/reports/coverage")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
test.finalizedBy jacocoTestReport
|
||||||
|
|
||||||
|
jacocoTestCoverageVerification {
|
||||||
|
violationRules {
|
||||||
|
rule {
|
||||||
|
limit {
|
||||||
|
counter = 'LINE'
|
||||||
|
minimum = 0.3
|
||||||
|
}
|
||||||
|
limit {
|
||||||
|
counter = 'BRANCH'
|
||||||
|
minimum = 0.3
|
||||||
|
}
|
||||||
|
limit {
|
||||||
|
counter = 'CLASS'
|
||||||
|
minimum = 0.4
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
check.dependsOn jacocoTestCoverageVerification
|
||||||
|
|
||||||
ospackage {
|
ospackage {
|
||||||
packageName = 'syslogd'
|
packageName = 'syslogd'
|
||||||
release = '1'
|
release = '1'
|
||||||
|
@ -80,5 +120,9 @@ jar {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
sourceCompatibility = 1.8
|
tasks.register("packages") {
|
||||||
targetCompatibility = 1.8
|
group "build"
|
||||||
|
dependsOn ":build"
|
||||||
|
dependsOn ":buildDeb"
|
||||||
|
dependsOn ":buildRpm"
|
||||||
|
}
|
||||||
|
|
|
@ -8,7 +8,7 @@ More information about the AIX errlogger is available on the IBM [knowledge cent
|
||||||
|
|
||||||
### Prepare the local syslog service
|
### Prepare the local syslog service
|
||||||
|
|
||||||
Configure the local syslog service to forward messages to our remote [syslogd](https://bitbucket.org/mnellemann/syslogd/) service.
|
Configure the local syslog service to forward messages to our remote [syslogd](https://git.data.coop/nellemann/syslogd/) service.
|
||||||
|
|
||||||
Create an empty local log file:
|
Create an empty local log file:
|
||||||
|
|
||||||
|
@ -39,7 +39,7 @@ refresh -s syslogd
|
||||||
|
|
||||||
### Forward errlogger to the local syslog
|
### Forward errlogger to the local syslog
|
||||||
|
|
||||||
We configure the errloger to forward messages to the local syslog service.
|
We configure the AIX [error logger](https://www.ibm.com/docs/en/aix/7.3?topic=concepts-error-logging-overview) to forward messages to the local syslog service.
|
||||||
|
|
||||||
Create an odm errnotify logging template file:
|
Create an odm errnotify logging template file:
|
||||||
|
|
||||||
|
|
|
@ -1,16 +1,20 @@
|
||||||
# Syslogd as a System Service
|
# Syslogd as a system service
|
||||||
|
|
||||||
## Systemd
|
## For systemd
|
||||||
|
|
||||||
Edit the **syslogd.service** and configure required options.
|
To install as a systemd service, copy the [syslogd.service](syslogd.service)
|
||||||
|
file into */etc/systemd/system/*, edit the file and configure your required options.
|
||||||
|
|
||||||
To install as a systemd service, copy the **syslogd.service**
|
Enable and start the service:
|
||||||
file into */etc/systemd/system/* and enable the service:
|
|
||||||
|
|
||||||
systemctl daemon-reload
|
```shell
|
||||||
systemctl enable syslogd.service
|
systemctl daemon-reload
|
||||||
systemctl restart syslogd.service
|
systemctl enable syslogd.service
|
||||||
|
systemctl restart syslogd.service
|
||||||
|
```
|
||||||
|
|
||||||
To read log output from the service, use:
|
To read log output from the service, use:
|
||||||
|
|
||||||
journalctl -f -u syslogd.service
|
```shell
|
||||||
|
journalctl -f -u syslogd.service
|
||||||
|
```
|
||||||
|
|
|
@ -0,0 +1 @@
|
||||||
|
<mxfile host="drawio-plugin" modified="2023-02-05T13:04:08.556Z" agent="5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.102 Safari/537.36" etag="__kq9uG-1g-sjP8t85Xj" version="20.5.3" type="embed"><diagram id="23iRSUPoRavnBvh4doch" name="Page-1"><mxGraphModel dx="809" dy="749" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="1169" pageHeight="827" math="0" shadow="0"><root><mxCell id="0"/><mxCell id="1" parent="0"/><mxCell id="10" style="edgeStyle=orthogonalEdgeStyle;orthogonalLoop=1;jettySize=auto;html=1;curved=1;sketch=1;shadow=1;" parent="1" source="2" target="3" edge="1"><mxGeometry relative="1" as="geometry"/></mxCell><mxCell id="2" value="Syslog<br>RFC--3164" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#ffe6cc;strokeColor=#d79b00;" parent="1" vertex="1"><mxGeometry x="50" y="90" width="120" height="60" as="geometry"/></mxCell><mxCell id="13" style="edgeStyle=orthogonalEdgeStyle;curved=1;sketch=1;orthogonalLoop=1;jettySize=auto;html=1;shadow=1;" parent="1" source="3" target="9" edge="1"><mxGeometry relative="1" as="geometry"/></mxCell><mxCell id="14" style="edgeStyle=orthogonalEdgeStyle;curved=1;sketch=1;orthogonalLoop=1;jettySize=auto;html=1;shadow=1;" parent="1" source="3" target="6" edge="1"><mxGeometry relative="1" as="geometry"/></mxCell><mxCell id="15" style="edgeStyle=orthogonalEdgeStyle;curved=1;sketch=1;orthogonalLoop=1;jettySize=auto;html=1;shadow=1;" parent="1" source="3" target="7" edge="1"><mxGeometry relative="1" as="geometry"/></mxCell><mxCell id="16" style="edgeStyle=orthogonalEdgeStyle;curved=1;sketch=1;orthogonalLoop=1;jettySize=auto;html=1;shadow=1;" parent="1" source="3" target="8" edge="1"><mxGeometry relative="1" as="geometry"/></mxCell><mxCell id="3" value="syslogd" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;sketch=1;rounded=1;shadow=1;fillColor=#dae8fc;strokeColor=#6c8ebf;" parent="1" vertex="1"><mxGeometry x="280" y="130" width="120" height="60" as="geometry"/></mxCell><mxCell id="11" style="edgeStyle=orthogonalEdgeStyle;curved=1;sketch=1;orthogonalLoop=1;jettySize=auto;html=1;shadow=1;" parent="1" source="4" target="3" edge="1"><mxGeometry relative="1" as="geometry"/></mxCell><mxCell id="4" value="Syslog<br>RFC--5424" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#fff2cc;strokeColor=#d6b656;" parent="1" vertex="1"><mxGeometry x="40" y="180" width="120" height="60" as="geometry"/></mxCell><mxCell id="12" style="edgeStyle=orthogonalEdgeStyle;curved=1;sketch=1;orthogonalLoop=1;jettySize=auto;html=1;shadow=1;" parent="1" source="5" target="3" edge="1"><mxGeometry relative="1" as="geometry"/></mxCell><mxCell id="5" value="GELF" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#f8cecc;strokeColor=#b85450;" parent="1" vertex="1"><mxGeometry x="120" y="260" width="120" height="60" as="geometry"/></mxCell><mxCell id="6" value="Syslog<br>RFC-5424" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#d5e8d4;strokeColor=#82b366;" parent="1" vertex="1"><mxGeometry x="570" y="80" width="120" height="60" as="geometry"/></mxCell><mxCell id="7" value="Grafana Loki" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#e1d5e7;strokeColor=#9673a6;" parent="1" vertex="1"><mxGeometry x="550" y="170" width="120" height="60" as="geometry"/></mxCell><mxCell id="8" value="Graylog" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#fff2cc;strokeColor=#d6b656;" parent="1" vertex="1"><mxGeometry x="420" y="250" width="120" height="60" as="geometry"/></mxCell><mxCell id="9" value="Standard<br>Output" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#f5f5f5;strokeColor=#666666;fontColor=#333333;" parent="1" vertex="1"><mxGeometry x="410" y="30" width="120" height="60" as="geometry"/></mxCell><mxCell id="17" value="Standard<br>Input" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#f5f5f5;strokeColor=#666666;fontColor=#333333;" vertex="1" parent="1"><mxGeometry x="180" y="30" width="90" height="60" as="geometry"/></mxCell><mxCell id="19" style="edgeStyle=orthogonalEdgeStyle;orthogonalLoop=1;jettySize=auto;html=1;curved=1;sketch=1;shadow=1;" edge="1" parent="1" source="17" target="3"><mxGeometry relative="1" as="geometry"><mxPoint x="190" y="110" as="sourcePoint"/><mxPoint x="300" y="170" as="targetPoint"/></mxGeometry></mxCell></root></mxGraphModel></diagram></mxfile>
|
Binary file not shown.
After Width: | Height: | Size: 96 KiB |
|
@ -1,10 +1,10 @@
|
||||||
[Unit]
|
[Unit]
|
||||||
Description=Simple Syslog Service
|
Description=Syslog Director
|
||||||
|
|
||||||
[Service]
|
[Service]
|
||||||
TimeoutStartSec=0
|
TimeoutStartSec=0
|
||||||
Restart=always
|
Restart=always
|
||||||
ExecStart=/opt/syslogd/bin/syslogd --no-stdout --syslog=udp://localhost:1514
|
ExecStart=/opt/syslogd/bin/syslogd --port 514 --no-ansi
|
||||||
|
|
||||||
[Install]
|
[Install]
|
||||||
WantedBy=default.target
|
WantedBy=default.target
|
||||||
|
|
|
@ -1,3 +1,5 @@
|
||||||
id = syslogd
|
id = syslogd
|
||||||
|
name = syslogd
|
||||||
group = biz.nellemann.syslogd
|
group = biz.nellemann.syslogd
|
||||||
version = 1.2.6
|
version = 1.3.5
|
||||||
|
description = "Syslog Director"
|
||||||
|
|
Binary file not shown.
|
@ -1,5 +1,6 @@
|
||||||
distributionBase=GRADLE_USER_HOME
|
distributionBase=GRADLE_USER_HOME
|
||||||
distributionPath=wrapper/dists
|
distributionPath=wrapper/dists
|
||||||
distributionUrl=https\://services.gradle.org/distributions/gradle-7.4-bin.zip
|
distributionUrl=https\://services.gradle.org/distributions/gradle-7.6.2-bin.zip
|
||||||
|
networkTimeout=10000
|
||||||
zipStoreBase=GRADLE_USER_HOME
|
zipStoreBase=GRADLE_USER_HOME
|
||||||
zipStorePath=wrapper/dists
|
zipStorePath=wrapper/dists
|
||||||
|
|
|
@ -55,7 +55,7 @@
|
||||||
# Darwin, MinGW, and NonStop.
|
# Darwin, MinGW, and NonStop.
|
||||||
#
|
#
|
||||||
# (3) This script is generated from the Groovy template
|
# (3) This script is generated from the Groovy template
|
||||||
# https://github.com/gradle/gradle/blob/master/subprojects/plugins/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt
|
# https://github.com/gradle/gradle/blob/HEAD/subprojects/plugins/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt
|
||||||
# within the Gradle project.
|
# within the Gradle project.
|
||||||
#
|
#
|
||||||
# You can find Gradle at https://github.com/gradle/gradle/.
|
# You can find Gradle at https://github.com/gradle/gradle/.
|
||||||
|
@ -80,10 +80,10 @@ do
|
||||||
esac
|
esac
|
||||||
done
|
done
|
||||||
|
|
||||||
APP_HOME=$( cd "${APP_HOME:-./}" && pwd -P ) || exit
|
# This is normally unused
|
||||||
|
# shellcheck disable=SC2034
|
||||||
APP_NAME="Gradle"
|
|
||||||
APP_BASE_NAME=${0##*/}
|
APP_BASE_NAME=${0##*/}
|
||||||
|
APP_HOME=$( cd "${APP_HOME:-./}" && pwd -P ) || exit
|
||||||
|
|
||||||
# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
|
# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
|
||||||
DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
|
DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
|
||||||
|
@ -143,12 +143,16 @@ fi
|
||||||
if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then
|
if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then
|
||||||
case $MAX_FD in #(
|
case $MAX_FD in #(
|
||||||
max*)
|
max*)
|
||||||
|
# In POSIX sh, ulimit -H is undefined. That's why the result is checked to see if it worked.
|
||||||
|
# shellcheck disable=SC3045
|
||||||
MAX_FD=$( ulimit -H -n ) ||
|
MAX_FD=$( ulimit -H -n ) ||
|
||||||
warn "Could not query maximum file descriptor limit"
|
warn "Could not query maximum file descriptor limit"
|
||||||
esac
|
esac
|
||||||
case $MAX_FD in #(
|
case $MAX_FD in #(
|
||||||
'' | soft) :;; #(
|
'' | soft) :;; #(
|
||||||
*)
|
*)
|
||||||
|
# In POSIX sh, ulimit -n is undefined. That's why the result is checked to see if it worked.
|
||||||
|
# shellcheck disable=SC3045
|
||||||
ulimit -n "$MAX_FD" ||
|
ulimit -n "$MAX_FD" ||
|
||||||
warn "Could not set maximum file descriptor limit to $MAX_FD"
|
warn "Could not set maximum file descriptor limit to $MAX_FD"
|
||||||
esac
|
esac
|
||||||
|
@ -205,6 +209,12 @@ set -- \
|
||||||
org.gradle.wrapper.GradleWrapperMain \
|
org.gradle.wrapper.GradleWrapperMain \
|
||||||
"$@"
|
"$@"
|
||||||
|
|
||||||
|
# Stop when "xargs" is not available.
|
||||||
|
if ! command -v xargs >/dev/null 2>&1
|
||||||
|
then
|
||||||
|
die "xargs is not available"
|
||||||
|
fi
|
||||||
|
|
||||||
# Use "xargs" to parse quoted args.
|
# Use "xargs" to parse quoted args.
|
||||||
#
|
#
|
||||||
# With -n1 it outputs one arg per line, with the quotes and backslashes removed.
|
# With -n1 it outputs one arg per line, with the quotes and backslashes removed.
|
||||||
|
|
|
@ -14,7 +14,7 @@
|
||||||
@rem limitations under the License.
|
@rem limitations under the License.
|
||||||
@rem
|
@rem
|
||||||
|
|
||||||
@if "%DEBUG%" == "" @echo off
|
@if "%DEBUG%"=="" @echo off
|
||||||
@rem ##########################################################################
|
@rem ##########################################################################
|
||||||
@rem
|
@rem
|
||||||
@rem Gradle startup script for Windows
|
@rem Gradle startup script for Windows
|
||||||
|
@ -25,7 +25,8 @@
|
||||||
if "%OS%"=="Windows_NT" setlocal
|
if "%OS%"=="Windows_NT" setlocal
|
||||||
|
|
||||||
set DIRNAME=%~dp0
|
set DIRNAME=%~dp0
|
||||||
if "%DIRNAME%" == "" set DIRNAME=.
|
if "%DIRNAME%"=="" set DIRNAME=.
|
||||||
|
@rem This is normally unused
|
||||||
set APP_BASE_NAME=%~n0
|
set APP_BASE_NAME=%~n0
|
||||||
set APP_HOME=%DIRNAME%
|
set APP_HOME=%DIRNAME%
|
||||||
|
|
||||||
|
@ -40,7 +41,7 @@ if defined JAVA_HOME goto findJavaFromJavaHome
|
||||||
|
|
||||||
set JAVA_EXE=java.exe
|
set JAVA_EXE=java.exe
|
||||||
%JAVA_EXE% -version >NUL 2>&1
|
%JAVA_EXE% -version >NUL 2>&1
|
||||||
if "%ERRORLEVEL%" == "0" goto execute
|
if %ERRORLEVEL% equ 0 goto execute
|
||||||
|
|
||||||
echo.
|
echo.
|
||||||
echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
|
echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
|
||||||
|
@ -75,13 +76,15 @@ set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar
|
||||||
|
|
||||||
:end
|
:end
|
||||||
@rem End local scope for the variables with windows NT shell
|
@rem End local scope for the variables with windows NT shell
|
||||||
if "%ERRORLEVEL%"=="0" goto mainEnd
|
if %ERRORLEVEL% equ 0 goto mainEnd
|
||||||
|
|
||||||
:fail
|
:fail
|
||||||
rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of
|
rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of
|
||||||
rem the _cmd.exe /c_ return code!
|
rem the _cmd.exe /c_ return code!
|
||||||
if not "" == "%GRADLE_EXIT_CONSOLE%" exit 1
|
set EXIT_CODE=%ERRORLEVEL%
|
||||||
exit /b 1
|
if %EXIT_CODE% equ 0 set EXIT_CODE=1
|
||||||
|
if not ""=="%GRADLE_EXIT_CONSOLE%" exit %EXIT_CODE%
|
||||||
|
exit /b %EXIT_CODE%
|
||||||
|
|
||||||
:mainEnd
|
:mainEnd
|
||||||
if "%OS%"=="Windows_NT" endlocal
|
if "%OS%"=="Windows_NT" endlocal
|
||||||
|
|
|
@ -1,10 +1 @@
|
||||||
/*
|
|
||||||
* This file was generated by the Gradle 'init' task.
|
|
||||||
*
|
|
||||||
* The settings file is used to specify which projects to include in your build.
|
|
||||||
*
|
|
||||||
* Detailed information about configuring a multi-project build in Gradle can be found
|
|
||||||
* in the user manual at https://docs.gradle.org/6.6.1/userguide/multi_project_builds.html
|
|
||||||
*/
|
|
||||||
|
|
||||||
rootProject.name = 'syslogd'
|
rootProject.name = 'syslogd'
|
||||||
|
|
|
@ -15,16 +15,6 @@
|
||||||
*/
|
*/
|
||||||
package biz.nellemann.syslogd;
|
package biz.nellemann.syslogd;
|
||||||
|
|
||||||
import biz.nellemann.syslogd.msg.SyslogMessage;
|
|
||||||
import biz.nellemann.syslogd.net.*;
|
|
||||||
import biz.nellemann.syslogd.parser.SyslogParser;
|
|
||||||
import biz.nellemann.syslogd.parser.SyslogParserRfc3164;
|
|
||||||
import biz.nellemann.syslogd.parser.SyslogParserRfc5424;
|
|
||||||
import org.slf4j.impl.SimpleLogger;
|
|
||||||
|
|
||||||
import picocli.CommandLine;
|
|
||||||
import picocli.CommandLine.Command;
|
|
||||||
|
|
||||||
import java.io.IOException;
|
import java.io.IOException;
|
||||||
import java.net.InetSocketAddress;
|
import java.net.InetSocketAddress;
|
||||||
import java.net.URI;
|
import java.net.URI;
|
||||||
|
@ -34,6 +24,19 @@ import java.util.List;
|
||||||
import java.util.Locale;
|
import java.util.Locale;
|
||||||
import java.util.concurrent.Callable;
|
import java.util.concurrent.Callable;
|
||||||
|
|
||||||
|
import biz.nellemann.syslogd.msg.SyslogMessage;
|
||||||
|
import biz.nellemann.syslogd.net.GelfClient;
|
||||||
|
import biz.nellemann.syslogd.net.LokiClient;
|
||||||
|
import biz.nellemann.syslogd.net.TcpServer;
|
||||||
|
import biz.nellemann.syslogd.net.UdpClient;
|
||||||
|
import biz.nellemann.syslogd.net.UdpServer;
|
||||||
|
import biz.nellemann.syslogd.parser.GelfParser;
|
||||||
|
import biz.nellemann.syslogd.parser.SyslogParser;
|
||||||
|
import biz.nellemann.syslogd.parser.SyslogParserRfc3164;
|
||||||
|
import biz.nellemann.syslogd.parser.SyslogParserRfc5424;
|
||||||
|
import picocli.CommandLine;
|
||||||
|
import picocli.CommandLine.Command;
|
||||||
|
|
||||||
@Command(name = "syslogd",
|
@Command(name = "syslogd",
|
||||||
mixinStandardHelpOptions = true,
|
mixinStandardHelpOptions = true,
|
||||||
versionProvider = biz.nellemann.syslogd.VersionProvider.class)
|
versionProvider = biz.nellemann.syslogd.VersionProvider.class)
|
||||||
|
@ -43,7 +46,7 @@ public class Application implements Callable<Integer>, LogReceiveListener {
|
||||||
private SyslogParser syslogParser;
|
private SyslogParser syslogParser;
|
||||||
|
|
||||||
|
|
||||||
@CommandLine.Option(names = {"-p", "--port"}, description = "Listening port [default: 514].", defaultValue = "514", paramLabel = "<num>")
|
@CommandLine.Option(names = {"-p", "--port"}, description = "Listening port [default: 1514].", defaultValue = "1514", paramLabel = "<num>")
|
||||||
private int port;
|
private int port;
|
||||||
|
|
||||||
@CommandLine.Option(names = "--no-udp", negatable = true, description = "Listen on UDP [default: true].", defaultValue = "true")
|
@CommandLine.Option(names = "--no-udp", negatable = true, description = "Listen on UDP [default: true].", defaultValue = "true")
|
||||||
|
@ -52,22 +55,25 @@ public class Application implements Callable<Integer>, LogReceiveListener {
|
||||||
@CommandLine.Option(names = "--no-tcp", negatable = true, description = "Listen on TCP [default: true].", defaultValue = "true")
|
@CommandLine.Option(names = "--no-tcp", negatable = true, description = "Listen on TCP [default: true].", defaultValue = "true")
|
||||||
private boolean tcpServer;
|
private boolean tcpServer;
|
||||||
|
|
||||||
@CommandLine.Option(names = "--no-ansi", negatable = true, description = "Output ANSI colors [default: true].", defaultValue = "true")
|
@CommandLine.Option(names = "--no-ansi", negatable = true, description = "Output in ANSI colors [default: true].", defaultValue = "true")
|
||||||
private boolean ansiOutput;
|
private boolean ansiOutput;
|
||||||
|
|
||||||
@CommandLine.Option(names = "--no-stdout", negatable = true, description = "Output messages to stdout [default: true].", defaultValue = "true")
|
@CommandLine.Option(names = "--no-stdout", negatable = true, description = "Output messages to stdout [default: true].", defaultValue = "true")
|
||||||
private boolean stdout;
|
private boolean stdout;
|
||||||
|
|
||||||
@CommandLine.Option(names = "--rfc5424", description = "Parse RFC-5424 messages [default: RFC-3164].", defaultValue = "false")
|
@CommandLine.Option(names = "--no-stdin", negatable = true, description = "Forward messages from stdin [default: true].", defaultValue = "true")
|
||||||
private boolean rfc5424;
|
private boolean stdin;
|
||||||
|
|
||||||
@CommandLine.Option(names = { "-s", "--syslog"}, description = "Forward to Syslog <udp://host:port> (RFC-5424).", paramLabel = "<uri>")
|
@CommandLine.Option(names = {"-f", "--format"}, description = "Input format: RFC-5424, RFC-3164 or GELF [default: RFC-3164].", defaultValue = "RFC-3164")
|
||||||
|
private String protocol;
|
||||||
|
|
||||||
|
@CommandLine.Option(names = { "--to-syslog"}, description = "Forward to Syslog <udp://host:port> (RFC-5424).", paramLabel = "<uri>")
|
||||||
private URI syslog;
|
private URI syslog;
|
||||||
|
|
||||||
@CommandLine.Option(names = { "-g", "--gelf"}, description = "Forward to Graylog <udp://host:port>.", paramLabel = "<uri>")
|
@CommandLine.Option(names = { "--to-gelf"}, description = "Forward to Graylog <udp://host:port>.", paramLabel = "<uri>")
|
||||||
private URI gelf;
|
private URI gelf;
|
||||||
|
|
||||||
@CommandLine.Option(names = { "-l", "--loki"}, description = "Forward to Grafana Loki <http://host:port>.", paramLabel = "<url>")
|
@CommandLine.Option(names = { "--to-loki"}, description = "Forward to Grafana Loki <http://host:port>.", paramLabel = "<url>")
|
||||||
private URL loki;
|
private URL loki;
|
||||||
|
|
||||||
@CommandLine.Option(names = { "-d", "--debug" }, description = "Enable debugging [default: 'false'].")
|
@CommandLine.Option(names = { "-d", "--debug" }, description = "Enable debugging [default: 'false'].")
|
||||||
|
@ -77,12 +83,13 @@ public class Application implements Callable<Integer>, LogReceiveListener {
|
||||||
@Override
|
@Override
|
||||||
public Integer call() throws IOException {
|
public Integer call() throws IOException {
|
||||||
|
|
||||||
|
|
||||||
if(enableDebug) {
|
if(enableDebug) {
|
||||||
System.setProperty(SimpleLogger.DEFAULT_LOG_LEVEL_KEY, "DEBUG");
|
System.setProperty("org.slf4j.simpleLogger.defaultLogLevel", "DEBUG");
|
||||||
}
|
}
|
||||||
|
|
||||||
if(rfc5424) {
|
if(protocol.equalsIgnoreCase("GELF"))
|
||||||
|
syslogParser = new GelfParser();
|
||||||
|
else if (protocol.equalsIgnoreCase("RFC-5424")) {
|
||||||
syslogParser = new SyslogParserRfc5424();
|
syslogParser = new SyslogParserRfc5424();
|
||||||
} else {
|
} else {
|
||||||
syslogParser = new SyslogParserRfc3164();
|
syslogParser = new SyslogParserRfc3164();
|
||||||
|
@ -113,6 +120,12 @@ public class Application implements Callable<Integer>, LogReceiveListener {
|
||||||
t.start();
|
t.start();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if(stdin) {
|
||||||
|
InputReader inputReader = new InputReader(System.in, protocol);
|
||||||
|
inputReader.addEventListener(this);
|
||||||
|
inputReader.start();
|
||||||
|
}
|
||||||
|
|
||||||
if(udpServer) {
|
if(udpServer) {
|
||||||
UdpServer udpServer = new UdpServer(port);
|
UdpServer udpServer = new UdpServer(port);
|
||||||
udpServer.addEventListener(this);
|
udpServer.addEventListener(this);
|
||||||
|
@ -133,17 +146,16 @@ public class Application implements Callable<Integer>, LogReceiveListener {
|
||||||
public void onLogEvent(LogReceiveEvent event) {
|
public void onLogEvent(LogReceiveEvent event) {
|
||||||
|
|
||||||
// Parse message
|
// Parse message
|
||||||
String message = event.getMessage();
|
|
||||||
SyslogMessage msg = null;
|
SyslogMessage msg = null;
|
||||||
try {
|
try {
|
||||||
msg = syslogParser.parse(message);
|
msg = syslogParser.parse(event.getBytes());
|
||||||
} catch(Exception e) {
|
} catch(Exception e) {
|
||||||
e.printStackTrace();
|
e.printStackTrace();
|
||||||
}
|
}
|
||||||
|
|
||||||
if(msg != null) {
|
if(msg != null) {
|
||||||
|
|
||||||
if(logForwardListeners.size() > 0) {
|
if(!logForwardListeners.isEmpty()) {
|
||||||
sendForwardEvent(msg);
|
sendForwardEvent(msg);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -169,8 +181,7 @@ public class Application implements Callable<Integer>, LogReceiveListener {
|
||||||
|
|
||||||
|
|
||||||
private InetSocketAddress getInetSocketAddress(URI input) {
|
private InetSocketAddress getInetSocketAddress(URI input) {
|
||||||
InetSocketAddress inetSocketAddress = new InetSocketAddress(input.getHost(), input.getPort());
|
return new InetSocketAddress(input.getHost(), input.getPort());
|
||||||
return inetSocketAddress;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|
|
@ -0,0 +1,70 @@
|
||||||
|
package biz.nellemann.syslogd;
|
||||||
|
|
||||||
|
import java.io.InputStream;
|
||||||
|
import java.util.ArrayList;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Scanner;
|
||||||
|
|
||||||
|
import biz.nellemann.syslogd.msg.SyslogMessage;
|
||||||
|
|
||||||
|
public class InputReader extends Thread {
|
||||||
|
|
||||||
|
private final Scanner input;
|
||||||
|
private final String protocol;
|
||||||
|
|
||||||
|
public InputReader(InputStream inputStream, String protocol) {
|
||||||
|
input = new Scanner(inputStream);
|
||||||
|
this.protocol = protocol;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void run() {
|
||||||
|
|
||||||
|
while(input.hasNextLine()) {
|
||||||
|
SyslogMessage msg = new SyslogMessage(input.nextLine());
|
||||||
|
msg.hostname = "localhost";
|
||||||
|
msg.application = "syslogd";
|
||||||
|
|
||||||
|
String payload;
|
||||||
|
if(protocol.equalsIgnoreCase("GELF"))
|
||||||
|
payload = SyslogPrinter.toGelf(msg);
|
||||||
|
else if (protocol.equalsIgnoreCase("RFC-5424")) {
|
||||||
|
payload = SyslogPrinter.toRfc5424(msg);
|
||||||
|
} else {
|
||||||
|
payload = SyslogPrinter.toRfc3164(msg);
|
||||||
|
}
|
||||||
|
|
||||||
|
sendEvent(payload);
|
||||||
|
}
|
||||||
|
input.close();
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
private synchronized void sendEvent(String text) {
|
||||||
|
LogReceiveEvent event = new LogReceiveEvent( this, text);
|
||||||
|
for (LogReceiveListener eventListener : eventListeners) {
|
||||||
|
eventListener.onLogEvent(event);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Event Listener Configuration
|
||||||
|
*/
|
||||||
|
|
||||||
|
protected List<LogReceiveListener> eventListeners = new ArrayList<>();
|
||||||
|
|
||||||
|
public synchronized void addEventListener(LogReceiveListener listener ) {
|
||||||
|
eventListeners.add( listener );
|
||||||
|
}
|
||||||
|
|
||||||
|
public synchronized void addEventListener(List<LogReceiveListener> listeners ) {
|
||||||
|
eventListeners.addAll(listeners);
|
||||||
|
}
|
||||||
|
|
||||||
|
public synchronized void removeEventListener( LogReceiveListener l ) {
|
||||||
|
eventListeners.remove( l );
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
}
|
|
@ -15,23 +15,33 @@
|
||||||
*/
|
*/
|
||||||
package biz.nellemann.syslogd;
|
package biz.nellemann.syslogd;
|
||||||
|
|
||||||
|
import java.net.DatagramPacket;
|
||||||
import biz.nellemann.syslogd.msg.SyslogMessage;
|
import java.nio.charset.StandardCharsets;
|
||||||
|
import java.util.Arrays;
|
||||||
import java.util.EventObject;
|
import java.util.EventObject;
|
||||||
|
|
||||||
public class LogReceiveEvent extends EventObject {
|
public class LogReceiveEvent extends EventObject {
|
||||||
|
|
||||||
private static final long serialVersionUID = 1L;
|
private static final long serialVersionUID = 1L;
|
||||||
private final String message;
|
private final DatagramPacket packet;
|
||||||
|
|
||||||
public LogReceiveEvent(final Object source, final String message ) {
|
public LogReceiveEvent(final Object source, final String message ) {
|
||||||
super( source );
|
super( source );
|
||||||
this.message = message;
|
byte[] bytes = message.getBytes();
|
||||||
|
this.packet = new DatagramPacket(bytes, bytes.length);
|
||||||
}
|
}
|
||||||
|
|
||||||
public String getMessage() {
|
public LogReceiveEvent(final Object source, final DatagramPacket packet) {
|
||||||
return message;
|
super( source );
|
||||||
|
this.packet = packet;
|
||||||
|
}
|
||||||
|
|
||||||
|
public String getText() {
|
||||||
|
return new String(packet.getData(), packet.getOffset(), packet.getLength(), StandardCharsets.UTF_8);
|
||||||
|
}
|
||||||
|
|
||||||
|
public byte[] getBytes() {
|
||||||
|
return Arrays.copyOfRange(packet.getData(), packet.getOffset(), packet.getLength());
|
||||||
}
|
}
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
|
@ -103,18 +103,18 @@ public class SyslogPrinter {
|
||||||
* @return
|
* @return
|
||||||
*/
|
*/
|
||||||
public static String toGelf(SyslogMessage msg) {
|
public static String toGelf(SyslogMessage msg) {
|
||||||
StringBuilder sb = new StringBuilder("{ \"version\": \"1.1\",");
|
StringBuilder sb = new StringBuilder("{ \"version\": \"1.1\"");
|
||||||
sb.append(String.format("\"host\": \"%s\",", msg.hostname));
|
sb.append(String.format(", \"host\": \"%s\"", msg.hostname));
|
||||||
sb.append(String.format("\"short_message\": \"%s\",", JsonUtil.encode(msg.message)));
|
sb.append(String.format(", \"short_message\": \"%s\"", JsonUtil.encode(msg.message)));
|
||||||
//sb.append(String.format("\"full_message\": \"%s\",", msg.message));
|
sb.append(String.format(", \"full_message\": \"%s\"", JsonUtil.encode(msg.structuredData)));
|
||||||
sb.append(String.format("\"timestamp\": %d,", msg.timestamp.getEpochSecond()));
|
sb.append(String.format(", \"timestamp\": %d", msg.timestamp.getEpochSecond()));
|
||||||
sb.append(String.format("\"level\": %d,", msg.severity.toNumber()));
|
sb.append(String.format(", \"level\": %d", msg.severity.toNumber()));
|
||||||
sb.append(String.format("\"_facility\": \"%s\",", msg.facility));
|
sb.append(String.format(", \"_facility\": \"%s\"", msg.facility));
|
||||||
sb.append(String.format("\"_severity\": \"%s\",", msg.severity));
|
sb.append(String.format(", \"_severity\": \"%s\"", msg.severity));
|
||||||
sb.append(String.format("\"_application\": \"%s\",", msg.application));
|
sb.append(String.format(", \"_application\": \"%s\"", msg.application));
|
||||||
if(msg.processId != null) { sb.append(String.format("\"_process-id\": \"%s\",", msg.processId)); }
|
if(msg.processId != null) { sb.append(String.format(", \"_process-id\": \"%s\"", msg.processId)); }
|
||||||
if(msg.messageId != null) { sb.append(String.format("\"_message-id\": \"%s\",", msg.messageId)); }
|
if(msg.messageId != null) { sb.append(String.format(", \"_message-id\": \"%s\"", msg.messageId)); }
|
||||||
if(msg.structuredData != null) { sb.append(String.format("\"_structured-data\": \"%s\",", msg.structuredData)); }
|
if(msg.structuredData != null) { sb.append(String.format(", \"_structured-data\": \"%s\"", JsonUtil.encode(msg.structuredData))); }
|
||||||
sb.append("}");
|
sb.append("}");
|
||||||
return sb.toString();
|
return sb.toString();
|
||||||
}
|
}
|
||||||
|
@ -133,7 +133,7 @@ public class SyslogPrinter {
|
||||||
sb.append(String.format(" \"level\": \"%s\",", msg.severity));
|
sb.append(String.format(" \"level\": \"%s\",", msg.severity));
|
||||||
sb.append(String.format(" \"application\": \"%s\"", msg.application));
|
sb.append(String.format(" \"application\": \"%s\"", msg.application));
|
||||||
sb.append("}, \"values\": [ ");
|
sb.append("}, \"values\": [ ");
|
||||||
sb.append(String.format("[ \"%d\", \"%s\" ]", msg.timestamp.getEpochSecond() * 1000000000l, getMessageLine(msg)));
|
sb.append(String.format("[ \"%d\", \"%s\" ]", msg.timestamp.getEpochSecond() * 1000000000L, getMessageLine(msg)));
|
||||||
sb.append(" ] } ] }");
|
sb.append(" ] } ] }");
|
||||||
log.debug(sb.toString());
|
log.debug(sb.toString());
|
||||||
return sb.toString();
|
return sb.toString();
|
||||||
|
|
|
@ -15,38 +15,55 @@
|
||||||
*/
|
*/
|
||||||
package biz.nellemann.syslogd.msg;
|
package biz.nellemann.syslogd.msg;
|
||||||
|
|
||||||
|
import com.fasterxml.jackson.annotation.JsonCreator;
|
||||||
|
import com.fasterxml.jackson.annotation.JsonIgnore;
|
||||||
|
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
|
||||||
|
import com.fasterxml.jackson.annotation.JsonProperty;
|
||||||
|
|
||||||
import java.time.Instant;
|
import java.time.Instant;
|
||||||
|
|
||||||
|
@JsonIgnoreProperties(ignoreUnknown = true)
|
||||||
public class SyslogMessage {
|
public class SyslogMessage {
|
||||||
|
|
||||||
public Facility facility;
|
@JsonIgnore
|
||||||
public Severity severity;
|
public Facility facility = Facility.user;
|
||||||
|
|
||||||
|
@JsonProperty("level")
|
||||||
|
public Severity severity = Severity.info;
|
||||||
|
|
||||||
// The VERSION field denotes the version of the syslog protocol specification.
|
// The VERSION field denotes the version of the syslog protocol specification.
|
||||||
public Integer version;
|
public String version;
|
||||||
|
|
||||||
// The TIMESTAMP field is a formalized timestamp derived from [RFC3339].
|
// The TIMESTAMP field is a formalized timestamp derived from [RFC3339].
|
||||||
public Instant timestamp;
|
@JsonProperty("timestamp") // 1670357783.694 - in GELF: seconds since UNIX epoch with optional decimal places for milliseconds
|
||||||
|
public Instant timestamp = Instant.now();
|
||||||
|
|
||||||
// The HOSTNAME field identifies the machine that originally sent the syslog message.
|
// The HOSTNAME field identifies the machine that originally sent the syslog message.
|
||||||
|
@JsonProperty("host")
|
||||||
public String hostname;
|
public String hostname;
|
||||||
|
|
||||||
// The APP-NAME field SHOULD identify the device or application that originated the message.
|
// The APP-NAME field SHOULD identify the device or application that originated the message.
|
||||||
|
@JsonProperty("_logger_name")
|
||||||
public String application;
|
public String application;
|
||||||
|
|
||||||
// The PROCID field is often used to provide the process name or process ID associated with a syslog system.
|
// The PROCID field is often used to provide the process name or process ID associated with a syslog system.
|
||||||
|
@JsonProperty("_thread_name")
|
||||||
public String processId;
|
public String processId;
|
||||||
|
|
||||||
// The MSGID SHOULD identify the type of message.
|
// The MSGID SHOULD identify the type of message.
|
||||||
|
@JsonIgnore
|
||||||
public String messageId;
|
public String messageId;
|
||||||
|
|
||||||
// STRUCTURED-DATA provides a mechanism to express information in a well defined, easily parseable and interpretable data format.
|
// STRUCTURED-DATA provides a mechanism to express information in a well defined, easily parseable and interpretable data format.
|
||||||
|
@JsonProperty("full_message")
|
||||||
public String structuredData;
|
public String structuredData;
|
||||||
|
|
||||||
// The MSG part contains a free-form message that provides information about the event.
|
// The MSG part contains a free-form message that provides information about the event.
|
||||||
public final String message;
|
@JsonProperty("short_message")
|
||||||
|
public String message;
|
||||||
|
|
||||||
public SyslogMessage(final String message) {
|
@JsonCreator
|
||||||
|
public SyslogMessage(@JsonProperty("short_message") final String message) {
|
||||||
this.message = message;
|
this.message = message;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
@ -1,13 +1,14 @@
|
||||||
package biz.nellemann.syslogd.net;
|
package biz.nellemann.syslogd.net;
|
||||||
|
|
||||||
import biz.nellemann.syslogd.LogForwardEvent;
|
|
||||||
import biz.nellemann.syslogd.SyslogPrinter;
|
|
||||||
import org.slf4j.Logger;
|
|
||||||
import org.slf4j.LoggerFactory;
|
|
||||||
|
|
||||||
import java.net.InetSocketAddress;
|
import java.net.InetSocketAddress;
|
||||||
import java.net.SocketException;
|
import java.net.SocketException;
|
||||||
|
|
||||||
|
import org.slf4j.Logger;
|
||||||
|
import org.slf4j.LoggerFactory;
|
||||||
|
|
||||||
|
import biz.nellemann.syslogd.LogForwardEvent;
|
||||||
|
import biz.nellemann.syslogd.SyslogPrinter;
|
||||||
|
|
||||||
public class GelfClient extends UdpClient {
|
public class GelfClient extends UdpClient {
|
||||||
|
|
||||||
private final static Logger log = LoggerFactory.getLogger(GelfClient.class);
|
private final static Logger log = LoggerFactory.getLogger(GelfClient.class);
|
||||||
|
|
|
@ -15,26 +15,28 @@
|
||||||
*/
|
*/
|
||||||
package biz.nellemann.syslogd.net;
|
package biz.nellemann.syslogd.net;
|
||||||
|
|
||||||
import biz.nellemann.syslogd.LogForwardEvent;
|
|
||||||
import biz.nellemann.syslogd.LogForwardListener;
|
|
||||||
import biz.nellemann.syslogd.SyslogPrinter;
|
|
||||||
import org.slf4j.Logger;
|
|
||||||
import org.slf4j.LoggerFactory;
|
|
||||||
|
|
||||||
import java.io.IOException;
|
import java.io.IOException;
|
||||||
import java.io.InputStream;
|
import java.io.InputStream;
|
||||||
import java.io.OutputStream;
|
import java.io.OutputStream;
|
||||||
import java.net.*;
|
import java.net.HttpURLConnection;
|
||||||
|
import java.net.MalformedURLException;
|
||||||
|
import java.net.URL;
|
||||||
import java.nio.charset.StandardCharsets;
|
import java.nio.charset.StandardCharsets;
|
||||||
import java.util.concurrent.ArrayBlockingQueue;
|
import java.util.concurrent.ArrayBlockingQueue;
|
||||||
|
|
||||||
|
import org.slf4j.Logger;
|
||||||
|
import org.slf4j.LoggerFactory;
|
||||||
|
|
||||||
|
import biz.nellemann.syslogd.LogForwardEvent;
|
||||||
|
import biz.nellemann.syslogd.LogForwardListener;
|
||||||
|
import biz.nellemann.syslogd.SyslogPrinter;
|
||||||
|
|
||||||
public class LokiClient implements LogForwardListener, Runnable {
|
public class LokiClient implements LogForwardListener, Runnable {
|
||||||
|
|
||||||
private final static Logger log = LoggerFactory.getLogger(LokiClient.class);
|
private final static Logger log = LoggerFactory.getLogger(LokiClient.class);
|
||||||
|
|
||||||
private final ArrayBlockingQueue<String> blockingQueue = new ArrayBlockingQueue<>(1024);
|
private final ArrayBlockingQueue<String> blockingQueue = new ArrayBlockingQueue<>(1024);
|
||||||
private final URL url;
|
private final URL url;
|
||||||
private boolean keepRunning = true;
|
|
||||||
|
|
||||||
|
|
||||||
public LokiClient(URL url) {
|
public LokiClient(URL url) {
|
||||||
|
@ -82,10 +84,10 @@ public class LokiClient implements LogForwardListener, Runnable {
|
||||||
@Override
|
@Override
|
||||||
public void run() {
|
public void run() {
|
||||||
|
|
||||||
while (keepRunning) {
|
while (true) {
|
||||||
try {
|
try {
|
||||||
send(blockingQueue.take());
|
send(blockingQueue.take());
|
||||||
} catch (Exception e) {
|
} catch (MalformedURLException | InterruptedException e) {
|
||||||
log.warn(e.getMessage());
|
log.warn(e.getMessage());
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
@ -21,20 +21,24 @@ import biz.nellemann.syslogd.LogReceiveListener;
|
||||||
import java.io.BufferedReader;
|
import java.io.BufferedReader;
|
||||||
import java.io.IOException;
|
import java.io.IOException;
|
||||||
import java.io.InputStreamReader;
|
import java.io.InputStreamReader;
|
||||||
|
import java.net.DatagramPacket;
|
||||||
import java.net.ServerSocket;
|
import java.net.ServerSocket;
|
||||||
import java.net.Socket;
|
import java.net.Socket;
|
||||||
|
import java.nio.charset.StandardCharsets;
|
||||||
import java.util.List;
|
import java.util.List;
|
||||||
|
|
||||||
|
import org.slf4j.Logger;
|
||||||
|
import org.slf4j.LoggerFactory;
|
||||||
|
|
||||||
import java.util.ArrayList;
|
import java.util.ArrayList;
|
||||||
|
|
||||||
public class TcpServer {
|
public class TcpServer {
|
||||||
|
|
||||||
|
private final static Logger log = LoggerFactory.getLogger(TcpServer.class);
|
||||||
|
|
||||||
private final int port;
|
private final int port;
|
||||||
private ServerSocket serverSocket;
|
private ServerSocket serverSocket;
|
||||||
|
|
||||||
public TcpServer() {
|
|
||||||
this(514);
|
|
||||||
}
|
|
||||||
|
|
||||||
public TcpServer(int port) {
|
public TcpServer(int port) {
|
||||||
this.port = port;
|
this.port = port;
|
||||||
}
|
}
|
||||||
|
@ -84,6 +88,8 @@ public class TcpServer {
|
||||||
|
|
||||||
public void run() {
|
public void run() {
|
||||||
|
|
||||||
|
// GELF TCP does not support compression due to the use of the null byte (\0) as frame delimiter.
|
||||||
|
// Is \0 also used as frame delimiter for regular syslog messages ?
|
||||||
try {
|
try {
|
||||||
in = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
|
in = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
|
||||||
String inputLine;
|
String inputLine;
|
||||||
|
@ -91,21 +97,22 @@ public class TcpServer {
|
||||||
sendEvent(inputLine);
|
sendEvent(inputLine);
|
||||||
}
|
}
|
||||||
} catch (IOException e) {
|
} catch (IOException e) {
|
||||||
e.printStackTrace();
|
log.warn("run() - read error: {}", e.getMessage());
|
||||||
}
|
}
|
||||||
|
|
||||||
try {
|
try {
|
||||||
in.close();
|
in.close();
|
||||||
clientSocket.close();
|
clientSocket.close();
|
||||||
} catch (IOException e) {
|
} catch (IOException e) {
|
||||||
e.printStackTrace();
|
log.warn("run() - close error: {}", e.getMessage());
|
||||||
}
|
}
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
private synchronized void sendEvent(String message) {
|
private synchronized void sendEvent(String message) {
|
||||||
LogReceiveEvent event = new LogReceiveEvent( this, message );
|
DatagramPacket packet = new DatagramPacket(message.getBytes(StandardCharsets.UTF_8), message.length());
|
||||||
|
LogReceiveEvent event = new LogReceiveEvent( this, packet);
|
||||||
for (LogReceiveListener eventListener : eventListeners) {
|
for (LogReceiveListener eventListener : eventListeners) {
|
||||||
eventListener.onLogEvent(event);
|
eventListener.onLogEvent(event);
|
||||||
}
|
}
|
||||||
|
|
|
@ -29,8 +29,8 @@ public class UdpClient implements LogForwardListener {
|
||||||
|
|
||||||
private final static Logger log = LoggerFactory.getLogger(UdpClient.class);
|
private final static Logger log = LoggerFactory.getLogger(UdpClient.class);
|
||||||
|
|
||||||
private InetSocketAddress inetSocketAddress;
|
private final InetSocketAddress inetSocketAddress;
|
||||||
private DatagramSocket socket;
|
private final DatagramSocket socket;
|
||||||
|
|
||||||
public UdpClient(InetSocketAddress inetSocketAddress) throws SocketException {
|
public UdpClient(InetSocketAddress inetSocketAddress) throws SocketException {
|
||||||
this.inetSocketAddress = inetSocketAddress;
|
this.inetSocketAddress = inetSocketAddress;
|
||||||
|
@ -40,12 +40,10 @@ public class UdpClient implements LogForwardListener {
|
||||||
public void send(String msg) {
|
public void send(String msg) {
|
||||||
byte[] buf = msg.getBytes(StandardCharsets.US_ASCII);
|
byte[] buf = msg.getBytes(StandardCharsets.US_ASCII);
|
||||||
DatagramPacket packet = new DatagramPacket(buf, buf.length, inetSocketAddress.getAddress(), inetSocketAddress.getPort());
|
DatagramPacket packet = new DatagramPacket(buf, buf.length, inetSocketAddress.getAddress(), inetSocketAddress.getPort());
|
||||||
if(this.socket != null) {
|
try {
|
||||||
try {
|
socket.send(packet);
|
||||||
socket.send(packet);
|
} catch (IOException e) {
|
||||||
} catch (IOException e) {
|
log.error("send() - Could not send packet: " + e.getMessage());
|
||||||
log.error("send() - Could not send packet: " + e.getMessage());
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
@ -15,54 +15,63 @@
|
||||||
*/
|
*/
|
||||||
package biz.nellemann.syslogd.net;
|
package biz.nellemann.syslogd.net;
|
||||||
|
|
||||||
import biz.nellemann.syslogd.LogReceiveEvent;
|
|
||||||
import biz.nellemann.syslogd.LogReceiveListener;
|
|
||||||
|
|
||||||
import java.io.IOException;
|
import java.io.IOException;
|
||||||
import java.net.DatagramPacket;
|
import java.net.DatagramPacket;
|
||||||
import java.net.DatagramSocket;
|
import java.net.DatagramSocket;
|
||||||
import java.nio.charset.StandardCharsets;
|
|
||||||
import java.util.ArrayList;
|
import java.util.ArrayList;
|
||||||
import java.util.List;
|
import java.util.List;
|
||||||
|
|
||||||
|
import org.slf4j.Logger;
|
||||||
|
import org.slf4j.LoggerFactory;
|
||||||
|
|
||||||
|
import biz.nellemann.syslogd.LogReceiveEvent;
|
||||||
|
import biz.nellemann.syslogd.LogReceiveListener;
|
||||||
|
|
||||||
public class UdpServer extends Thread {
|
public class UdpServer extends Thread {
|
||||||
|
|
||||||
|
private final static Logger log = LoggerFactory.getLogger(UdpServer.class);
|
||||||
|
|
||||||
protected DatagramSocket socket;
|
protected DatagramSocket socket;
|
||||||
protected boolean listen = true;
|
protected boolean listen = true;
|
||||||
|
|
||||||
public UdpServer() throws IOException {
|
|
||||||
this(514);
|
|
||||||
}
|
|
||||||
|
|
||||||
public UdpServer(int port) throws IOException {
|
public UdpServer(int port) throws IOException {
|
||||||
socket = new DatagramSocket(port);
|
socket = new DatagramSocket(port);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
public void run() {
|
public void run() {
|
||||||
|
|
||||||
byte[] buf = new byte[4096];
|
byte[] buf = new byte[8192];
|
||||||
DatagramPacket packet = new DatagramPacket(buf, buf.length);
|
DatagramPacket packet = new DatagramPacket(buf, buf.length);
|
||||||
|
|
||||||
while (listen) {
|
while (listen) {
|
||||||
try {
|
try {
|
||||||
socket.receive(packet);
|
socket.receive(packet);
|
||||||
String packetData = new String(packet.getData(), packet.getOffset(), packet.getLength(), StandardCharsets.UTF_8);
|
//String packetData = new String(packet.getData(), packet.getOffset(), packet.getLength(), StandardCharsets.UTF_8);
|
||||||
sendEvent(packetData);
|
sendEvent(packet);
|
||||||
} catch (Exception e) {
|
} catch (IOException e) {
|
||||||
e.printStackTrace();
|
log.error("run() - error: {}", e.getMessage());
|
||||||
listen = false;
|
listen = false;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
socket.close();
|
socket.close();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/*
|
||||||
private synchronized void sendEvent(String message) {
|
private synchronized void sendEvent(String message) {
|
||||||
LogReceiveEvent event = new LogReceiveEvent( this, message);
|
LogReceiveEvent event = new LogReceiveEvent( this, message);
|
||||||
for (LogReceiveListener eventListener : eventListeners) {
|
for (LogReceiveListener eventListener : eventListeners) {
|
||||||
eventListener.onLogEvent(event);
|
eventListener.onLogEvent(event);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
*/
|
||||||
|
|
||||||
|
private synchronized void sendEvent(DatagramPacket packet) {
|
||||||
|
LogReceiveEvent event = new LogReceiveEvent( this, packet);
|
||||||
|
for (LogReceiveListener eventListener : eventListeners) {
|
||||||
|
eventListener.onLogEvent(event);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Event Listener Configuration
|
* Event Listener Configuration
|
||||||
|
|
|
@ -0,0 +1,136 @@
|
||||||
|
package biz.nellemann.syslogd.parser;
|
||||||
|
|
||||||
|
import java.time.Instant;
|
||||||
|
import java.util.Arrays;
|
||||||
|
import java.util.TreeMap;
|
||||||
|
|
||||||
|
import org.apache.commons.collections4.map.PassiveExpiringMap;
|
||||||
|
import org.slf4j.Logger;
|
||||||
|
import org.slf4j.LoggerFactory;
|
||||||
|
|
||||||
|
import com.fasterxml.jackson.core.JsonProcessingException;
|
||||||
|
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||||
|
import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule;
|
||||||
|
|
||||||
|
import biz.nellemann.syslogd.msg.SyslogMessage;
|
||||||
|
|
||||||
|
/*
|
||||||
|
For more information about the GELF format, visit: https://go2docs.graylog.org/5-0/getting_in_log_data/gelf.html
|
||||||
|
*/
|
||||||
|
public class GelfParser extends SyslogParser {
|
||||||
|
|
||||||
|
private final static Logger log = LoggerFactory.getLogger(GelfParser.class);
|
||||||
|
|
||||||
|
private final ObjectMapper objectMapper;
|
||||||
|
|
||||||
|
private final int expiryInMills = 10_000;
|
||||||
|
private final PassiveExpiringMap<Integer, TreeMap<Integer, byte[]>> expiringMap = new PassiveExpiringMap<>(expiryInMills);
|
||||||
|
|
||||||
|
|
||||||
|
public GelfParser() {
|
||||||
|
objectMapper = new ObjectMapper();
|
||||||
|
objectMapper.registerModule(new JavaTimeModule());
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
/*
|
||||||
|
Magic Bytes - 2 bytes: 0x1e 0x0f
|
||||||
|
Message ID - 8 bytes: Must be the same for every chunk of this message.
|
||||||
|
Identifies the whole message and is used to reassemble the chunks later.
|
||||||
|
Generate from millisecond timestamp + hostname, for example.
|
||||||
|
Sequence number - 1 byte: The sequence number of this chunk starts at 0 and is always less than the sequence count.
|
||||||
|
Sequence count - 1 byte: Total number of chunks this message has.
|
||||||
|
|
||||||
|
All chunks MUST arrive within 5 seconds or the server will discard all chunks that have arrived or are in the process of arriving.
|
||||||
|
A message MUST NOT consist of more than 128 chunks.
|
||||||
|
*/
|
||||||
|
private SyslogMessage parseChunked(byte[] input) {
|
||||||
|
|
||||||
|
if(input.length < 12) return null;
|
||||||
|
|
||||||
|
byte[] messageId = { input[2], input[3], input[4], input[5], input[6], input[7], input[8], input[9] };
|
||||||
|
byte seqNumber = input[10];
|
||||||
|
byte seqTotal = input[11];
|
||||||
|
byte[] payload = Arrays.copyOfRange(input, 12, input.length);
|
||||||
|
log.debug("parseChunked() - msgId: {}, seqNo: {}, seqTot: {}, payload: {}", messageId, seqNumber, seqTotal, byteArrayToString(payload));
|
||||||
|
|
||||||
|
// messageId byte[] to int
|
||||||
|
int id = 0;
|
||||||
|
for (byte b : messageId) {
|
||||||
|
id = (id << 8) + (b & 0xFF);
|
||||||
|
}
|
||||||
|
|
||||||
|
TreeMap<Integer, byte[]> integerTreeMap;
|
||||||
|
if(expiringMap.containsKey(id)) {
|
||||||
|
integerTreeMap = expiringMap.get(id);
|
||||||
|
} else {
|
||||||
|
integerTreeMap = new TreeMap<>();
|
||||||
|
}
|
||||||
|
integerTreeMap.put((int)seqNumber, payload);
|
||||||
|
expiringMap.put(id, integerTreeMap);
|
||||||
|
|
||||||
|
if(integerTreeMap.size() >= seqTotal) {
|
||||||
|
StringBuilder sb = new StringBuilder();
|
||||||
|
integerTreeMap.forEach( (i, p) -> {
|
||||||
|
sb.append(byteArrayToString(p));
|
||||||
|
});
|
||||||
|
return parse(sb.toString());
|
||||||
|
}
|
||||||
|
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public SyslogMessage parse(String input) {
|
||||||
|
if(!input.startsWith("{")) return null; // Avoid trying to parse non-JSON content
|
||||||
|
SyslogMessage message = null;
|
||||||
|
try {
|
||||||
|
message = objectMapper.readValue(input, SyslogMessage.class);
|
||||||
|
} catch (JsonProcessingException e) {
|
||||||
|
log.debug("parse() - error: {}", e.getMessage());
|
||||||
|
}
|
||||||
|
return message;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
/*
|
||||||
|
zlib signatures at offset 0
|
||||||
|
78 01 : No Compression (no preset dictionary)
|
||||||
|
78 5E : Best speed (no preset dictionary)
|
||||||
|
78 9C : Default Compression (no preset dictionary)
|
||||||
|
78 DA : Best Compression (no preset dictionary)
|
||||||
|
78 20 : No Compression (with preset dictionary)
|
||||||
|
78 7D : Best speed (with preset dictionary)
|
||||||
|
78 BB : Default Compression (with preset dictionary)
|
||||||
|
78 F9 : Best Compression (with preset dictionary)
|
||||||
|
|
||||||
|
gzip signature at offset 0
|
||||||
|
1F 8B : GZIP compressed
|
||||||
|
*/
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public SyslogMessage parse(byte[] input) {
|
||||||
|
|
||||||
|
if(input.length < 8) return null; // TODO: Find proper minimum input length ?
|
||||||
|
|
||||||
|
// Compressed data: 0x78 0x9c
|
||||||
|
if(input[0] == (byte)0x78 && input[1] == (byte)0x9c) {
|
||||||
|
input = decompress(input);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Magic Bytes: 0x1e 0x0f
|
||||||
|
if(input[0] == (byte)0x1e && input[1] == (byte)0x0f) {
|
||||||
|
return parseChunked(input);
|
||||||
|
}
|
||||||
|
|
||||||
|
return parse(byteArrayToString(input));
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public Instant parseTimestamp(String dateString) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
}
|
|
@ -6,92 +6,96 @@ package biz.nellemann.syslogd.parser;
|
||||||
|
|
||||||
public class JsonUtil {
|
public class JsonUtil {
|
||||||
|
|
||||||
public static String encode(String input) {
|
public static String encode(String input) {
|
||||||
|
|
||||||
StringBuilder output = new StringBuilder();
|
if(input == null) {
|
||||||
|
return "";
|
||||||
for(int i=0; i<input.length(); i++) {
|
|
||||||
char ch = input.charAt(i);
|
|
||||||
int chx = (int) ch;
|
|
||||||
|
|
||||||
// let's not put any nulls in our strings
|
|
||||||
if(chx == 0) {
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
|
|
||||||
if(ch == '\n') {
|
|
||||||
output.append("\\n");
|
|
||||||
} else if(ch == '\t') {
|
|
||||||
output.append("\\t");
|
|
||||||
} else if(ch == '\r') {
|
|
||||||
output.append("\\r");
|
|
||||||
} else if(ch == '\\') {
|
|
||||||
output.append("\\\\");
|
|
||||||
} else if(ch == '"') {
|
|
||||||
output.append("\\\"");
|
|
||||||
} else if(ch == '\b') {
|
|
||||||
output.append("\\b");
|
|
||||||
} else if(ch == '\f') {
|
|
||||||
output.append("\\f");
|
|
||||||
} else if(chx > 127) {
|
|
||||||
output.append(String.format("\\u%04x", chx));
|
|
||||||
} else {
|
|
||||||
output.append(ch);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return output.toString();
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
public static String decode(String input) {
|
|
||||||
|
|
||||||
StringBuilder builder = new StringBuilder();
|
|
||||||
|
|
||||||
int i = 0;
|
|
||||||
while (i < input.length()) {
|
|
||||||
char delimiter = input.charAt(i); i++; // consume letter or backslash
|
|
||||||
|
|
||||||
if(delimiter == '\\' && i < input.length()) {
|
|
||||||
|
|
||||||
// consume first after backslash
|
|
||||||
char ch = input.charAt(i); i++;
|
|
||||||
|
|
||||||
if(ch == '\\' || ch == '/' || ch == '"' || ch == '\'') {
|
|
||||||
builder.append(ch);
|
|
||||||
}
|
}
|
||||||
else if(ch == 'n') builder.append('\n');
|
|
||||||
else if(ch == 'r') builder.append('\r');
|
|
||||||
else if(ch == 't') builder.append('\t');
|
|
||||||
else if(ch == 'b') builder.append('\b');
|
|
||||||
else if(ch == 'f') builder.append('\f');
|
|
||||||
else if(ch == 'u') {
|
|
||||||
|
|
||||||
StringBuilder hex = new StringBuilder();
|
StringBuilder output = new StringBuilder();
|
||||||
|
|
||||||
// expect 4 digits
|
for (int i = 0; i < input.length(); i++) {
|
||||||
if (i+4 > input.length()) {
|
char ch = input.charAt(i);
|
||||||
throw new RuntimeException("Not enough unicode digits! ");
|
|
||||||
}
|
// let's not put any nulls in our strings
|
||||||
for (char x : input.substring(i, i + 4).toCharArray()) {
|
if ((int) ch == 0) {
|
||||||
if(!Character.isLetterOrDigit(x)) {
|
continue;
|
||||||
throw new RuntimeException("Bad character in unicode escape.");
|
|
||||||
}
|
}
|
||||||
hex.append(Character.toLowerCase(x));
|
|
||||||
}
|
|
||||||
i+=4; // consume those four digits.
|
|
||||||
|
|
||||||
int code = Integer.parseInt(hex.toString(), 16);
|
if (ch == '\n') {
|
||||||
builder.append((char) code);
|
output.append("\\n");
|
||||||
} else {
|
} else if (ch == '\t') {
|
||||||
throw new RuntimeException("Illegal escape sequence: \\"+ch);
|
output.append("\\t");
|
||||||
|
} else if (ch == '\r') {
|
||||||
|
output.append("\\r");
|
||||||
|
} else if (ch == '\\') {
|
||||||
|
output.append("\\\\");
|
||||||
|
} else if (ch == '"') {
|
||||||
|
output.append("\\\"");
|
||||||
|
} else if (ch == '\b') {
|
||||||
|
output.append("\\b");
|
||||||
|
} else if (ch == '\f') {
|
||||||
|
output.append("\\f");
|
||||||
|
} else if ((int) ch > 127) {
|
||||||
|
output.append(String.format("\\u%04x", (int) ch));
|
||||||
|
} else {
|
||||||
|
output.append(ch);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
} else { // it's not a backslash, or it's the last character.
|
|
||||||
builder.append(delimiter);
|
return output.toString();
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
return builder.toString();
|
|
||||||
}
|
public static String decode(String input) {
|
||||||
|
|
||||||
|
StringBuilder builder = new StringBuilder();
|
||||||
|
|
||||||
|
int i = 0;
|
||||||
|
while (i < input.length()) {
|
||||||
|
char delimiter = input.charAt(i);
|
||||||
|
i++; // consume letter or backslash
|
||||||
|
|
||||||
|
if (delimiter == '\\' && i < input.length()) {
|
||||||
|
|
||||||
|
// consume first after backslash
|
||||||
|
char ch = input.charAt(i);
|
||||||
|
i++;
|
||||||
|
|
||||||
|
if (ch == '\\' || ch == '/' || ch == '"' || ch == '\'') {
|
||||||
|
builder.append(ch);
|
||||||
|
} else if (ch == 'n') builder.append('\n');
|
||||||
|
else if (ch == 'r') builder.append('\r');
|
||||||
|
else if (ch == 't') builder.append('\t');
|
||||||
|
else if (ch == 'b') builder.append('\b');
|
||||||
|
else if (ch == 'f') builder.append('\f');
|
||||||
|
else if (ch == 'u') {
|
||||||
|
|
||||||
|
StringBuilder hex = new StringBuilder();
|
||||||
|
|
||||||
|
// expect 4 digits
|
||||||
|
if (i + 4 > input.length()) {
|
||||||
|
throw new RuntimeException("Not enough unicode digits! ");
|
||||||
|
}
|
||||||
|
for (char x : input.substring(i, i + 4).toCharArray()) {
|
||||||
|
if (!Character.isLetterOrDigit(x)) {
|
||||||
|
throw new RuntimeException("Bad character in unicode escape.");
|
||||||
|
}
|
||||||
|
hex.append(Character.toLowerCase(x));
|
||||||
|
}
|
||||||
|
i += 4; // consume those four digits.
|
||||||
|
|
||||||
|
int code = Integer.parseInt(hex.toString(), 16);
|
||||||
|
builder.append((char) code);
|
||||||
|
} else {
|
||||||
|
throw new RuntimeException("Illegal escape sequence: \\" + ch);
|
||||||
|
}
|
||||||
|
} else { // it's not a backslash, or it's the last character.
|
||||||
|
builder.append(delimiter);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return builder.toString();
|
||||||
|
}
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
|
@ -15,18 +15,22 @@
|
||||||
*/
|
*/
|
||||||
package biz.nellemann.syslogd.parser;
|
package biz.nellemann.syslogd.parser;
|
||||||
|
|
||||||
import biz.nellemann.syslogd.msg.SyslogMessage;
|
import java.nio.charset.StandardCharsets;
|
||||||
|
import java.time.Instant;
|
||||||
|
import java.util.zip.DataFormatException;
|
||||||
|
import java.util.zip.Inflater;
|
||||||
|
|
||||||
import org.slf4j.Logger;
|
import org.slf4j.Logger;
|
||||||
import org.slf4j.LoggerFactory;
|
import org.slf4j.LoggerFactory;
|
||||||
|
|
||||||
import java.time.Instant;
|
import biz.nellemann.syslogd.msg.SyslogMessage;
|
||||||
|
|
||||||
public abstract class SyslogParser {
|
public abstract class SyslogParser {
|
||||||
|
|
||||||
private final static Logger log = LoggerFactory.getLogger(SyslogParser.class);
|
private final static Logger log = LoggerFactory.getLogger(SyslogParser.class);
|
||||||
|
|
||||||
|
|
||||||
public abstract SyslogMessage parse(final String input);
|
public abstract SyslogMessage parse(final String input);
|
||||||
|
public abstract SyslogMessage parse(final byte[] input);
|
||||||
|
|
||||||
public abstract Instant parseTimestamp(final String dateString);
|
public abstract Instant parseTimestamp(final String dateString);
|
||||||
|
|
||||||
|
@ -38,11 +42,8 @@ public abstract class SyslogParser {
|
||||||
* @return
|
* @return
|
||||||
*/
|
*/
|
||||||
public int getFacility(String pri) {
|
public int getFacility(String pri) {
|
||||||
|
|
||||||
int priority = Integer.parseInt(pri);
|
int priority = Integer.parseInt(pri);
|
||||||
int facility = priority >> 3;
|
int facility = priority >> 3;
|
||||||
|
|
||||||
//log.debug("getFacility() - " + pri + " => " + facility);
|
|
||||||
return facility;
|
return facility;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -54,13 +55,35 @@ public abstract class SyslogParser {
|
||||||
* @return
|
* @return
|
||||||
*/
|
*/
|
||||||
public int getSeverity(String pri) {
|
public int getSeverity(String pri) {
|
||||||
|
|
||||||
int priority = Integer.parseInt(pri);
|
int priority = Integer.parseInt(pri);
|
||||||
int severity = priority & 0x07;
|
int severity = priority & 0x07;
|
||||||
|
|
||||||
//log.debug("getSeverity() - " + pri + " => " + severity);
|
|
||||||
return severity;
|
return severity;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
protected String byteArrayToString(byte[] input) {
|
||||||
|
return new String(input, 0, input.length, StandardCharsets.UTF_8);
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
protected byte[] decompress(byte[] data) {
|
||||||
|
|
||||||
|
byte[] result = new byte[data.length * 2];
|
||||||
|
try {
|
||||||
|
// Decompress the bytes
|
||||||
|
Inflater decompressor = new Inflater();
|
||||||
|
decompressor.setInput(data, 0, data.length);
|
||||||
|
//byte[] result = new byte[data.length * 2];
|
||||||
|
decompressor.inflate(result);
|
||||||
|
decompressor.end();
|
||||||
|
|
||||||
|
// Decode the bytes into a String
|
||||||
|
//uncompressed = new String(result, 0, resultLength, StandardCharsets.UTF_8);
|
||||||
|
} catch (DataFormatException e) {
|
||||||
|
log.error("decompress() - error: {}", e.getMessage());
|
||||||
|
}
|
||||||
|
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
|
@ -15,23 +15,26 @@
|
||||||
*/
|
*/
|
||||||
package biz.nellemann.syslogd.parser;
|
package biz.nellemann.syslogd.parser;
|
||||||
|
|
||||||
import biz.nellemann.syslogd.msg.Facility;
|
import java.time.Instant;
|
||||||
import biz.nellemann.syslogd.msg.Severity;
|
import java.time.OffsetDateTime;
|
||||||
import biz.nellemann.syslogd.msg.SyslogMessage;
|
import java.time.ZoneId;
|
||||||
import org.slf4j.Logger;
|
|
||||||
import org.slf4j.LoggerFactory;
|
|
||||||
|
|
||||||
import java.time.*;
|
|
||||||
import java.time.format.DateTimeFormatter;
|
import java.time.format.DateTimeFormatter;
|
||||||
import java.time.format.DateTimeParseException;
|
import java.time.format.DateTimeParseException;
|
||||||
import java.util.regex.Matcher;
|
import java.util.regex.Matcher;
|
||||||
import java.util.regex.Pattern;
|
import java.util.regex.Pattern;
|
||||||
|
|
||||||
|
import org.slf4j.Logger;
|
||||||
|
import org.slf4j.LoggerFactory;
|
||||||
|
|
||||||
|
import biz.nellemann.syslogd.msg.Facility;
|
||||||
|
import biz.nellemann.syslogd.msg.Severity;
|
||||||
|
import biz.nellemann.syslogd.msg.SyslogMessage;
|
||||||
|
|
||||||
public class SyslogParserRfc3164 extends SyslogParser {
|
public class SyslogParserRfc3164 extends SyslogParser {
|
||||||
|
|
||||||
private final static Logger log = LoggerFactory.getLogger(SyslogParserRfc3164.class);
|
private final static Logger log = LoggerFactory.getLogger(SyslogParserRfc3164.class);
|
||||||
|
|
||||||
private final Pattern pattern = Pattern.compile("^<(\\d{1,3})>(\\D{3}\\s+\\d{1,2} \\d{2}:\\d{2}:\\d{2})\\s+(Message forwarded from \\S+:|\\S+)\\s+([^\\s:]+):?\\s+(.*)", Pattern.CASE_INSENSITIVE);
|
private final Pattern pattern = Pattern.compile("^<(\\d{1,3})>(\\D{3}\\s+\\d{1,2} \\d{2}:\\d{2}:\\d{2})\\s+(Message forwarded from \\S+:|\\S+:?)\\s+([^\\s:]+):?\\s+(.*)", Pattern.CASE_INSENSITIVE);
|
||||||
private final DateTimeFormatter dateTimeFormatter = DateTimeFormatter.ofPattern("yyyy MMM [ ]d HH:mm:ss").withZone(ZoneId.systemDefault());
|
private final DateTimeFormatter dateTimeFormatter = DateTimeFormatter.ofPattern("yyyy MMM [ ]d HH:mm:ss").withZone(ZoneId.systemDefault());
|
||||||
|
|
||||||
/**
|
/**
|
||||||
|
@ -77,6 +80,11 @@ public class SyslogParserRfc3164 extends SyslogParser {
|
||||||
return syslogMessage;
|
return syslogMessage;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public SyslogMessage parse(byte[] input) {
|
||||||
|
return parse(byteArrayToString(input));
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Parse rfc3164 TIMESTAMP field into Instant.
|
* Parse rfc3164 TIMESTAMP field into Instant.
|
||||||
|
@ -84,6 +92,7 @@ public class SyslogParserRfc3164 extends SyslogParser {
|
||||||
* @param dateString
|
* @param dateString
|
||||||
* @return
|
* @return
|
||||||
*/
|
*/
|
||||||
|
@Override
|
||||||
public Instant parseTimestamp(String dateString) {
|
public Instant parseTimestamp(String dateString) {
|
||||||
|
|
||||||
// We need to add current year to parse date correctly
|
// We need to add current year to parse date correctly
|
||||||
|
|
|
@ -15,27 +15,26 @@
|
||||||
*/
|
*/
|
||||||
package biz.nellemann.syslogd.parser;
|
package biz.nellemann.syslogd.parser;
|
||||||
|
|
||||||
import biz.nellemann.syslogd.msg.Severity;
|
|
||||||
import biz.nellemann.syslogd.msg.Facility;
|
|
||||||
import biz.nellemann.syslogd.msg.SyslogMessage;
|
|
||||||
import org.slf4j.Logger;
|
|
||||||
import org.slf4j.LoggerFactory;
|
|
||||||
|
|
||||||
import java.text.DateFormat;
|
|
||||||
import java.text.ParseException;
|
import java.text.ParseException;
|
||||||
import java.text.SimpleDateFormat;
|
import java.text.SimpleDateFormat;
|
||||||
import java.time.*;
|
import java.time.Instant;
|
||||||
import java.time.format.DateTimeFormatter;
|
|
||||||
import java.util.Arrays;
|
import java.util.Arrays;
|
||||||
import java.util.List;
|
import java.util.List;
|
||||||
import java.util.regex.Matcher;
|
import java.util.regex.Matcher;
|
||||||
import java.util.regex.Pattern;
|
import java.util.regex.Pattern;
|
||||||
|
|
||||||
|
import org.slf4j.Logger;
|
||||||
|
import org.slf4j.LoggerFactory;
|
||||||
|
|
||||||
|
import biz.nellemann.syslogd.msg.Facility;
|
||||||
|
import biz.nellemann.syslogd.msg.Severity;
|
||||||
|
import biz.nellemann.syslogd.msg.SyslogMessage;
|
||||||
|
|
||||||
public class SyslogParserRfc5424 extends SyslogParser {
|
public class SyslogParserRfc5424 extends SyslogParser {
|
||||||
|
|
||||||
private final static Logger log = LoggerFactory.getLogger(SyslogParserRfc5424.class);
|
private final static Logger log = LoggerFactory.getLogger(SyslogParserRfc5424.class);
|
||||||
|
|
||||||
private final Pattern pattern = Pattern.compile("^<(\\d{1,3})>(\\d+)\\s+(\\S+)\\s+(\\S+)\\s+(\\S+)\\s+(\\S+)\\s+(\\S+)\\s+(\\[.*\\]|-)\\s+(\\S+)", Pattern.CASE_INSENSITIVE);
|
private final Pattern pattern = Pattern.compile("^<(\\d{1,3})>(\\d+)\\s+(\\S+)\\s+(\\S+)\\s+(\\S+)\\s+(\\S+)\\s+(\\S+)\\s+(\\[.*\\]|-)\\s+(.*)", Pattern.CASE_INSENSITIVE);
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Parses [rfc5424](https://tools.ietf.org/html/rfc5424) syslog messages.
|
* Parses [rfc5424](https://tools.ietf.org/html/rfc5424) syslog messages.
|
||||||
|
@ -73,7 +72,7 @@ public class SyslogParserRfc5424 extends SyslogParser {
|
||||||
SyslogMessage syslogMessage = new SyslogMessage(msg.trim());
|
SyslogMessage syslogMessage = new SyslogMessage(msg.trim());
|
||||||
syslogMessage.facility = Facility.getByNumber(facility);
|
syslogMessage.facility = Facility.getByNumber(facility);
|
||||||
syslogMessage.severity = Severity.getByNumber(severity);
|
syslogMessage.severity = Severity.getByNumber(severity);
|
||||||
syslogMessage.version = Integer.parseInt(ver);
|
syslogMessage.version = ver;
|
||||||
syslogMessage.timestamp = parseTimestamp(date);
|
syslogMessage.timestamp = parseTimestamp(date);
|
||||||
syslogMessage.hostname = host;
|
syslogMessage.hostname = host;
|
||||||
if(app != null && !app.equals("-"))
|
if(app != null && !app.equals("-"))
|
||||||
|
@ -88,6 +87,11 @@ public class SyslogParserRfc5424 extends SyslogParser {
|
||||||
return syslogMessage;
|
return syslogMessage;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public SyslogMessage parse(byte[] input) {
|
||||||
|
return parse(byteArrayToString(input));
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Parse rfc5424 TIMESTAMP field into Instant.
|
* Parse rfc5424 TIMESTAMP field into Instant.
|
||||||
|
@ -95,6 +99,7 @@ public class SyslogParserRfc5424 extends SyslogParser {
|
||||||
* @param dateString
|
* @param dateString
|
||||||
* @return
|
* @return
|
||||||
*/
|
*/
|
||||||
|
@Override
|
||||||
public Instant parseTimestamp(String dateString) {
|
public Instant parseTimestamp(String dateString) {
|
||||||
|
|
||||||
/*
|
/*
|
||||||
|
|
|
@ -0,0 +1,100 @@
|
||||||
|
package biz.nellemann.syslogd
|
||||||
|
|
||||||
|
import biz.nellemann.syslogd.msg.Facility
|
||||||
|
import biz.nellemann.syslogd.msg.Severity
|
||||||
|
import biz.nellemann.syslogd.msg.SyslogMessage
|
||||||
|
import biz.nellemann.syslogd.parser.GelfParser
|
||||||
|
import biz.nellemann.syslogd.parser.SyslogParser
|
||||||
|
import spock.lang.Specification
|
||||||
|
|
||||||
|
|
||||||
|
class GelfParserTest extends Specification {
|
||||||
|
|
||||||
|
SyslogParser syslogParser;
|
||||||
|
|
||||||
|
void setup() {
|
||||||
|
syslogParser = new GelfParser();
|
||||||
|
}
|
||||||
|
|
||||||
|
void "uncompressed GELF message"() {
|
||||||
|
|
||||||
|
setup:
|
||||||
|
def input = '{"version":"1.1","host":"pop-os.localdomain","short_message":"main() - Starting VTD-Camera","full_message":"main() - Starting VTD-Camera\\n","timestamp":1670357783.694,"level":4,"_thread_name":"main","_logger_name":"vtd.camera.Application"}'
|
||||||
|
|
||||||
|
when:
|
||||||
|
SyslogMessage msg = syslogParser.parse(input)
|
||||||
|
|
||||||
|
then:
|
||||||
|
msg.version == "1.1"
|
||||||
|
msg.message == "main() - Starting VTD-Camera"
|
||||||
|
msg.hostname == "pop-os.localdomain"
|
||||||
|
msg.application == "vtd.camera.Application"
|
||||||
|
msg.processId == "main"
|
||||||
|
msg.timestamp.toString() == "2022-12-06T20:16:23.694Z"
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
void "compressed GELF message"() {
|
||||||
|
|
||||||
|
setup:
|
||||||
|
byte[] input = [ 120, -100, -51, 81, 77, 107, -61, 48, 12, -3, 43, -63, -25, -38, -115, -77, -75, -51, 2, -127, -11, -80, -61, 96, -73, 94, 11, -63, 56, 90, -30, -59, 31, -63, -106, -53, -54, -40, 127, -97, 92, 86, -40, 79, -40, 69, -78, -97, -11, -12, -98, -28, 47, 118, -127, -104, 76, -16, -84, 99, 82, 72, -74, 97, 115, 72, 72, 23, -77, 114, 89, 115, 73, 65, -14, 102, 87, 11, -56, 92, -125, -57, -88, 44, -105, 66, 7, -73, 102, 4, 97, 60, 66, -12, -54, 18, 45, -51, 33, -30, -32, 32, 37, 53, 1, -15, -31, 66, -43, -3, 49, -29, 76, -39, 104, -123, -92, 113, -54, 90, 83, -63, 75, 121, -86, 114, 42, 84, 7, -67, 83, 113, 17, 30, -84, 5, -89, -68, 127, -98, -100, 50, -74, 40, 84, 8, 94, 81, -113, -90, -118, -32, 2, -62, 113, 28, 35, -79, 123, -39, -18, -124, 108, -91, 104, -102, 7, 33, -27, -95, 74, 4, 82, -13, -41, -79, -9, -39, 22, 43, -17, -108, -2, -127, -109, -77, 39, 47, 104, -56, 8, 42, -73, -78, 78, -18, 15, -11, -82, -106, -121, -89, 86, -44, -113, -5, 13, -77, 100, -52, -78, -114, 78, -125, 90, -41, 65, -121, 76, -21, -67, -110, -31, 113, 97, -65, 88, 113, 69, -128, -93, 61, -57, -126, -39, 48, 77, 16, -17, -80, -6, 8, 57, -118, 99, -119, 39, -48, 57, 26, -68, -2, -99, -21, -51, 36, -14, 13, 55, 34, 77, 72, -1, 60, -28, 72, -126, 108, 70, 92, 83, 119, -34, -98, -73, -29, 34, 110, -67, 5, -119, -35, 53, -63, 95, -88, 102, -115, 97, 44, 8, -50, 17, -44, 120, 87, 44, 76, -18, 77, -32, 109, -35, -42, 28, 62, 65, -13, -122, 125, -1, 0, -40, 60, -57, -72 ];
|
||||||
|
|
||||||
|
when:
|
||||||
|
SyslogMessage msg = syslogParser.parse(input)
|
||||||
|
|
||||||
|
then:
|
||||||
|
msg.version == "1.1"
|
||||||
|
msg.facility == Facility.user;
|
||||||
|
msg.severity == Severity.info;
|
||||||
|
msg.message == "event=AuthenticationSuccessEvent username=mark.nellemann@gmail.com tenant=2 remoteAddress=185.181.223.117 sessionId=null"
|
||||||
|
msg.hostname == "ip-10-1-101-250.eu-central-1.compute.internal"
|
||||||
|
msg.application == "ajour.AjourSecuritySuccessEventListener"
|
||||||
|
msg.processId == "http-nio-8080-exec-2"
|
||||||
|
msg.timestamp.toString() == "2022-12-08T12:16:38.046Z"
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
void "chunked GELF message"() {
|
||||||
|
|
||||||
|
setup:
|
||||||
|
byte[] chunk1 = [30, 15, -103, 51, -96, -10, -51, -31, 39, -41, 0, 2, 123, 34, 118, 101, 114, 115, 105, 111, 110, 34, 58, 34, 49, 46, 49, 34, 44, 34, 104, 111, 115, 116, 34, 58, 34, 105, 112, 45, 49, 48, 45, 49, 45, 49, 48, 49, 45, 49, 52, 46, 101, 117, 45, 99, 101, 110, 116, 114, 97, 108, 45, 49, 46, 99, 111, 109, 112, 117, 116, 101, 46, 105, 110, 116, 101, 114, 110, 97, 108, 34, 44, 34, 115, 104, 111, 114, 116, 95, 109, 101, 115, 115, 97, 103, 101, 34, 58, 34, 101, 118, 101, 110, 116, 61, 65, 117, 116, 104, 101, 110, 116, 105, 99, 97, 116, 105, 111, 110, 83, 117, 99, 99, 101, 115, 115, 69, 118, 101, 110, 116, 32, 117, 115, 101, 114, 110, 97, 109, 101, 61, 109, 97, 114, 107, 46, 110, 101, 108, 108, 101, 109, 97, 110, 110, 64, 103, 109, 97, 105, 108, 46, 99, 111, 109, 32, 116, 101, 110, 97, 110, 116, 61, 50, 32, 114, 101, 109, 111, 116, 101, 65, 100, 100, 114, 101, 115, 115, 61, 49, 56, 53, 46, 49, 56, 49, 46, 50, 50, 51, 46, 49, 49, 55, 32, 115, 101, 115, 115, 105, 111, 110, 73, 100, 61, 110, 117, 108, 108, 34, 44, 34, 102, 117, 108, 108, 95, 109, 101, 115, 115, 97, 103, 101, 34, 58, 34, 101, 118, 101, 110, 116, 61, 65, 117, 116, 104, 101, 110, 116, 105, 99, 97, 116, 105, 111, 110, 83, 117, 99, 99, 101, 115, 115, 69, 118, 101, 110, 116, 32, 117, 115, 101, 114, 110, 97, 109, 101, 61, 109, 97, 114, 107, 46, 110, 101, 108, 108, 101, 109, 97, 110, 110, 64, 103, 109, 97, 105, 108, 46, 99, 111, 109, 32, 116, 101, 110, 97, 110, 116, 61, 50, 32, 114, 101, 109, 111, 116, 101, 65, 100, 100, 114, 101, 115, 115, 61, 49, 56, 53, 46, 49, 56, 49, 46, 50, 50, 51, 46, 49, 49, 55, 32, 115, 101, 115, 115, 105, 111, 110, 73, 100, 61, 110, 117, 108, 108, 92, 110, 34, 44, 34, 116, 105, 109, 101, 115, 116, 97, 109, 112, 34, 58, 49, 54, 55, 49, 48, 48, 48, 53, 53, 54, 46, 56, 48, 50, 44, 34, 108, 101, 118, 101, 108, 34, 58, 54, 44, 34, 95, 97, 112, 112, 95, 99, 111, 117, 110, 116, 114, 121, 34, 58, 34, 100, 107, 34, 44, 34, 95, 97, 112, 112, 95, 110, 97, 109, 101, 34, 58, 34, 109, 105, 110, 116, 114, 34, 44, 34, 95, 108, 111, 103, 103, 101, 114, 95, 110, 97, 109, 101, 34, 58, 34, 97, 106, 111, 117, 114, 46, 65, 106, 111, 117, 114, 83, 101, 99, 117, 114, 105, 116, 121, 83, 117, 99, 99, 101, 115, 115, 69, 118, 101, 110, 116, 76, 105, 115, 116, 101, 110, 101, 114, 34, 44, 34, 95, 115, 101, 114, 118, 101, 114, 95, 117, 114, 108]
|
||||||
|
byte[] chunk2 = [30, 15, -103, 51, -96, -10, -51, -31, 39, -41, 1, 2, 34, 58, 34, 104, 116, 116, 112, 115, 58, 92, 47, 92, 47, 100, 107, 46, 109, 105, 110, 116, 114, 46, 97, 112, 112, 34, 44, 34, 95, 97, 112, 112, 95, 101, 110, 118, 34, 58, 34, 112, 114, 111, 100, 34, 44, 34, 95, 116, 104, 114, 101, 97, 100, 95, 110, 97, 109, 101, 34, 58, 34, 104, 116, 116, 112, 45, 110, 105, 111, 45, 56, 48, 56, 48, 45, 101, 120, 101, 99, 45, 52, 34, 125]
|
||||||
|
|
||||||
|
when:
|
||||||
|
syslogParser.parse(chunk1)
|
||||||
|
SyslogMessage msg = syslogParser.parse(chunk2)
|
||||||
|
|
||||||
|
then:
|
||||||
|
msg.message == "event=AuthenticationSuccessEvent username=mark.nellemann@gmail.com tenant=2 remoteAddress=185.181.223.117 sessionId=null"
|
||||||
|
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
void "chunked GELF unordered message"() {
|
||||||
|
|
||||||
|
setup:
|
||||||
|
byte[] chunk1 = [30, 15, -103, 51, -96, -10, -51, -31, 39, -41, 0, 2, 123, 34, 118, 101, 114, 115, 105, 111, 110, 34, 58, 34, 49, 46, 49, 34, 44, 34, 104, 111, 115, 116, 34, 58, 34, 105, 112, 45, 49, 48, 45, 49, 45, 49, 48, 49, 45, 49, 52, 46, 101, 117, 45, 99, 101, 110, 116, 114, 97, 108, 45, 49, 46, 99, 111, 109, 112, 117, 116, 101, 46, 105, 110, 116, 101, 114, 110, 97, 108, 34, 44, 34, 115, 104, 111, 114, 116, 95, 109, 101, 115, 115, 97, 103, 101, 34, 58, 34, 101, 118, 101, 110, 116, 61, 65, 117, 116, 104, 101, 110, 116, 105, 99, 97, 116, 105, 111, 110, 83, 117, 99, 99, 101, 115, 115, 69, 118, 101, 110, 116, 32, 117, 115, 101, 114, 110, 97, 109, 101, 61, 109, 97, 114, 107, 46, 110, 101, 108, 108, 101, 109, 97, 110, 110, 64, 103, 109, 97, 105, 108, 46, 99, 111, 109, 32, 116, 101, 110, 97, 110, 116, 61, 50, 32, 114, 101, 109, 111, 116, 101, 65, 100, 100, 114, 101, 115, 115, 61, 49, 56, 53, 46, 49, 56, 49, 46, 50, 50, 51, 46, 49, 49, 55, 32, 115, 101, 115, 115, 105, 111, 110, 73, 100, 61, 110, 117, 108, 108, 34, 44, 34, 102, 117, 108, 108, 95, 109, 101, 115, 115, 97, 103, 101, 34, 58, 34, 101, 118, 101, 110, 116, 61, 65, 117, 116, 104, 101, 110, 116, 105, 99, 97, 116, 105, 111, 110, 83, 117, 99, 99, 101, 115, 115, 69, 118, 101, 110, 116, 32, 117, 115, 101, 114, 110, 97, 109, 101, 61, 109, 97, 114, 107, 46, 110, 101, 108, 108, 101, 109, 97, 110, 110, 64, 103, 109, 97, 105, 108, 46, 99, 111, 109, 32, 116, 101, 110, 97, 110, 116, 61, 50, 32, 114, 101, 109, 111, 116, 101, 65, 100, 100, 114, 101, 115, 115, 61, 49, 56, 53, 46, 49, 56, 49, 46, 50, 50, 51, 46, 49, 49, 55, 32, 115, 101, 115, 115, 105, 111, 110, 73, 100, 61, 110, 117, 108, 108, 92, 110, 34, 44, 34, 116, 105, 109, 101, 115, 116, 97, 109, 112, 34, 58, 49, 54, 55, 49, 48, 48, 48, 53, 53, 54, 46, 56, 48, 50, 44, 34, 108, 101, 118, 101, 108, 34, 58, 54, 44, 34, 95, 97, 112, 112, 95, 99, 111, 117, 110, 116, 114, 121, 34, 58, 34, 100, 107, 34, 44, 34, 95, 97, 112, 112, 95, 110, 97, 109, 101, 34, 58, 34, 109, 105, 110, 116, 114, 34, 44, 34, 95, 108, 111, 103, 103, 101, 114, 95, 110, 97, 109, 101, 34, 58, 34, 97, 106, 111, 117, 114, 46, 65, 106, 111, 117, 114, 83, 101, 99, 117, 114, 105, 116, 121, 83, 117, 99, 99, 101, 115, 115, 69, 118, 101, 110, 116, 76, 105, 115, 116, 101, 110, 101, 114, 34, 44, 34, 95, 115, 101, 114, 118, 101, 114, 95, 117, 114, 108]
|
||||||
|
byte[] chunk2 = [30, 15, -103, 51, -96, -10, -51, -31, 39, -41, 1, 2, 34, 58, 34, 104, 116, 116, 112, 115, 58, 92, 47, 92, 47, 100, 107, 46, 109, 105, 110, 116, 114, 46, 97, 112, 112, 34, 44, 34, 95, 97, 112, 112, 95, 101, 110, 118, 34, 58, 34, 112, 114, 111, 100, 34, 44, 34, 95, 116, 104, 114, 101, 97, 100, 95, 110, 97, 109, 101, 34, 58, 34, 104, 116, 116, 112, 45, 110, 105, 111, 45, 56, 48, 56, 48, 45, 101, 120, 101, 99, 45, 52, 34, 125]
|
||||||
|
|
||||||
|
when:
|
||||||
|
syslogParser.parse(chunk2)
|
||||||
|
SyslogMessage msg = syslogParser.parse(chunk1)
|
||||||
|
|
||||||
|
then:
|
||||||
|
msg.message == "event=AuthenticationSuccessEvent username=mark.nellemann@gmail.com tenant=2 remoteAddress=185.181.223.117 sessionId=null"
|
||||||
|
|
||||||
|
}
|
||||||
|
|
||||||
|
void "junk GET request"() {
|
||||||
|
|
||||||
|
setup:
|
||||||
|
def input = 'GET /'
|
||||||
|
|
||||||
|
when:
|
||||||
|
SyslogMessage msg = syslogParser.parse(input)
|
||||||
|
|
||||||
|
then:
|
||||||
|
msg == null
|
||||||
|
}
|
||||||
|
|
||||||
|
}
|
|
@ -39,4 +39,14 @@ class JsonUtilTest extends Specification {
|
||||||
result == 'here it comes " to wreck the day...'
|
result == 'here it comes " to wreck the day...'
|
||||||
}
|
}
|
||||||
|
|
||||||
|
def "test newline decode"() {
|
||||||
|
setup:
|
||||||
|
def testQuote = 'here it comes \n to wreck the day...'
|
||||||
|
|
||||||
|
when:
|
||||||
|
def result = JsonUtil.decode(testQuote)
|
||||||
|
|
||||||
|
then:
|
||||||
|
result == 'here it comes \n to wreck the day...'
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
|
@ -50,13 +50,13 @@ class SyslogParserRfc3164Test extends Specification {
|
||||||
void "test rfc3164 normal message"() {
|
void "test rfc3164 normal message"() {
|
||||||
|
|
||||||
setup:
|
setup:
|
||||||
def input = "<13>Sep 23 08:53:28 xps13 mark: adfdfdf3432434"
|
def input = "<13>Sep 23 08:53:28 xps13 mark: adfdfdf3432434 abcdefghijklmnopqrstuvwxyz"
|
||||||
|
|
||||||
when:
|
when:
|
||||||
SyslogMessage msg = syslogParser.parse(input)
|
SyslogMessage msg = syslogParser.parse(input)
|
||||||
|
|
||||||
then:
|
then:
|
||||||
msg.message == "adfdfdf3432434"
|
msg.message == "adfdfdf3432434 abcdefghijklmnopqrstuvwxyz"
|
||||||
msg.hostname == "xps13"
|
msg.hostname == "xps13"
|
||||||
msg.application == "mark"
|
msg.application == "mark"
|
||||||
}
|
}
|
||||||
|
|
|
@ -40,6 +40,7 @@ class SyslogParserRfc5424Test extends Specification {
|
||||||
msg.application == "su"
|
msg.application == "su"
|
||||||
msg.messageId == "ID47"
|
msg.messageId == "ID47"
|
||||||
msg.processId == null
|
msg.processId == null
|
||||||
|
msg.message == "BOM'su root' failed for lonvick on /dev/pts/8"
|
||||||
}
|
}
|
||||||
|
|
||||||
void "test rfc5424 example2 message"() {
|
void "test rfc5424 example2 message"() {
|
||||||
|
|
|
@ -10,10 +10,23 @@ class SyslogPrinterTest extends Specification {
|
||||||
void setup() {
|
void setup() {
|
||||||
}
|
}
|
||||||
|
|
||||||
|
void "to plain"() {
|
||||||
|
setup:
|
||||||
|
SyslogParser syslogParser = new SyslogParserRfc5424();
|
||||||
|
String input = '<13>1 2020-09-23T08:57:30.950699+02:00 xps13 mark - - [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"] adfdfdf3432434565656 abcdefghijklmnopqrstuvwxyz'
|
||||||
|
SyslogMessage msg = syslogParser.parse(input)
|
||||||
|
|
||||||
|
when:
|
||||||
|
String output = SyslogPrinter.toString(msg)
|
||||||
|
|
||||||
|
then:
|
||||||
|
output.endsWith("abcdefghijklmnopqrstuvwxyz")
|
||||||
|
}
|
||||||
|
|
||||||
void "test toGelf"() {
|
void "test toGelf"() {
|
||||||
setup:
|
setup:
|
||||||
SyslogParser syslogParser = new SyslogParserRfc5424();
|
SyslogParser syslogParser = new SyslogParserRfc5424();
|
||||||
String input = '<13>1 2020-09-23T08:57:30.950699+02:00 xps13 mark - - [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"] adfdfdf3432434565656'
|
String input = '<13>1 2020-09-23T08:57:30.950699+02:00 xps13 mark - - [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"] adfdfdf3432434565656 abcdefghijklmnopqrstuvwxyz'
|
||||||
SyslogMessage msg = syslogParser.parse(input)
|
SyslogMessage msg = syslogParser.parse(input)
|
||||||
|
|
||||||
when:
|
when:
|
||||||
|
@ -26,14 +39,14 @@ class SyslogPrinterTest extends Specification {
|
||||||
void "test toLoki"() {
|
void "test toLoki"() {
|
||||||
setup:
|
setup:
|
||||||
SyslogParser syslogParser = new SyslogParserRfc5424();
|
SyslogParser syslogParser = new SyslogParserRfc5424();
|
||||||
String input = '<13>1 2020-09-23T08:57:30.950699+02:00 xps13 mark - - [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"] adfdfdf3432434565656'
|
String input = '<13>1 2020-09-23T08:57:30.950699+02:00 xps13 mark - - [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"] adfdfdf3432434565656 abcdefghijklmnopqrstuvwxyz'
|
||||||
SyslogMessage msg = syslogParser.parse(input)
|
SyslogMessage msg = syslogParser.parse(input)
|
||||||
|
|
||||||
when:
|
when:
|
||||||
String output = SyslogPrinter.toLoki(msg)
|
String output = SyslogPrinter.toLoki(msg)
|
||||||
|
|
||||||
then:
|
then:
|
||||||
output == '{ "streams": [ { "stream": { "hostname": "xps13", "facility": "user", "level": "notice", "application": "mark"}, "values": [ [ "1600845200000000000", "[user.notice] xps13 mark adfdfdf3432434565656" ] ] } ] }'
|
output == '{ "streams": [ { "stream": { "hostname": "xps13", "facility": "user", "level": "notice", "application": "mark"}, "values": [ [ "1600845200000000000", "[user.notice] xps13 mark adfdfdf3432434565656 abcdefghijklmnopqrstuvwxyz" ] ] } ] }'
|
||||||
}
|
}
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
Loading…
Reference in New Issue