Compare commits

...

10 Commits
v1.3.1 ... main

Author SHA1 Message Date
Mark Nellemann 3ef961e44b Update README.md 2024-05-17 07:53:25 +00:00
Mark Nellemann bd5e2634d5 Merge pull request 'Depedency updates and cleanup.' (#1) from updates into main
continuous-integration/drone/push Build is passing Details
continuous-integration/drone/tag Build is passing Details
Reviewed-on: #1
2023-11-13 12:48:20 +00:00
Mark Nellemann 1acdd6a93d Cleanup.
continuous-integration/drone/push Build is passing Details
continuous-integration/drone/pr Build is passing Details
2023-11-13 13:46:11 +01:00
Mark Nellemann d39837861f Update dependencies and gradle. 2023-10-02 14:25:33 +02:00
Mark Nellemann 422f1fbb71 Read from stdin and forward.
continuous-integration/drone/push Build is passing Details
continuous-integration/drone/tag Build is passing Details
2023-02-05 14:38:54 +01:00
Mark Nellemann d3589faf9e Fix syslog message parsing error resulting in messages being cut at first whitespace.
continuous-integration/drone/push Build is passing Details
continuous-integration/drone/tag Build is passing Details
2023-01-25 08:47:31 +01:00
Mark Nellemann c47f682c34 Improve robustness of GELF forwarding and parsing.
continuous-integration/drone/push Build is passing Details
continuous-integration/drone/tag Build is passing Details
2023-01-22 11:44:03 +01:00
Mark Nellemann b291f87693 Update links and gradle.
continuous-integration/drone/push Build is passing Details
2023-01-06 08:06:52 +01:00
Mark Nellemann 3e7ba2f46e Cleanup and gradle update.
continuous-integration/drone/push Build is passing Details
2023-01-03 12:49:21 +01:00
Mark Nellemann a0ad98bf52 Update 'README.md'
continuous-integration/drone/push Build is passing Details
2023-01-03 08:24:18 +00:00
32 changed files with 271 additions and 241 deletions

View File

@ -11,13 +11,13 @@ steps:
- name: publish - name: publish
image: eclipse-temurin:8-jdk image: eclipse-temurin:8-jdk
environment: environment:
AUTH_TOKEN: AUTH_TOKEN: # Gitea access token ENV variable
from_secret: auth # gitea_username:gitea_token from_secret: auth # Name of DroneCI secret exposed above
commands: commands:
- ./gradlew packages - ./gradlew packages
- for file in build/libs/*-all.jar ; do curl --user "$${AUTH_TOKEN}" --upload-file "$${file}" "https://git.data.coop/api/packages/${DRONE_REPO_OWNER}/generic/${DRONE_REPO_NAME}/${DRONE_TAG}/$(basename $file)" ; done - for file in build/libs/*-all.jar ; do curl --user "${DRONE_REPO_OWNER}:$${AUTH_TOKEN}" --upload-file "$${file}" "https://git.data.coop/api/packages/${DRONE_REPO_OWNER}/generic/${DRONE_REPO_NAME}/${DRONE_TAG}/$(basename $file)" ; done
- for file in build/distributions/*.deb ; do curl --user "$${AUTH_TOKEN}" --upload-file "$${file}" "https://git.data.coop/api/packages/${DRONE_REPO_OWNER}/generic/${DRONE_REPO_NAME}/${DRONE_TAG}/$(basename $file)" ; done - for file in build/distributions/*.deb ; do curl --user "${DRONE_REPO_OWNER}:$${AUTH_TOKEN}" --upload-file "$${file}" "https://git.data.coop/api/packages/${DRONE_REPO_OWNER}/generic/${DRONE_REPO_NAME}/${DRONE_TAG}/$(basename $file)" ; done
- for file in build/distributions/*.rpm ; do curl --user "$${AUTH_TOKEN}" --upload-file "$${file}" "https://git.data.coop/api/packages/${DRONE_REPO_OWNER}/generic/${DRONE_REPO_NAME}/${DRONE_TAG}/$(basename $file)" ; done - for file in build/distributions/*.rpm ; do curl --user "${DRONE_REPO_OWNER}:$${AUTH_TOKEN}" --upload-file "$${file}" "https://git.data.coop/api/packages/${DRONE_REPO_OWNER}/generic/${DRONE_REPO_NAME}/${DRONE_TAG}/$(basename $file)" ; done
when: when:
event: event:
- tag - tag

108
README.md
View File

@ -1,107 +1,3 @@
# Syslog Director # Repository moved
All received messages are written to *stdout* and/or forwarded to remote logging destinations.
Supported incoming message formats are:
- Syslog RFC5424 - TCP and UDP
- Syslog RFC3164 (BSD) - TCP and UDP
- Graylog GELF - TCP and UDP (compressed & chunked)
Supported remote logging destinations are:
- Syslog (RFC5424 over UDP)
- Graylog (GELF over UDP)
- Grafana Loki (HTTP over TCP).
This software is free to use and is licensed under the [Apache 2.0 License](LICENSE).
![architecture](doc/syslogd.png)
Some of my other related projects are:
- [hmci](https://bitbucket.org/mnellemann/hmci) for agent-less monitoring of IBM Power servers
- [svci](https://bitbucket.org/mnellemann/svci) for monitoring IBM Spectrum Virtualize (Flashsystems / Storwize / SVC)
- [sysmon](https://bitbucket.org/mnellemann/sysmon) for monitoring all types of servers with a small Java agent
## Usage Instructions
- Install the syslogd package (*.deb* or *.rpm*) from [downloads](https://bitbucket.org/mnellemann/syslogd/downloads/) or build from source.
```text
Usage: syslogd [-dhV] [--[no-]ansi] [--[no-]stdout] [--[no-]tcp] [--[no-]udp]
[-f=<protocol>] [-p=<num>] [--to-gelf=<uri>] [--to-loki=<url>]
[--to-syslog=<uri>]
-d, --debug Enable debugging [default: 'false'].
-f, --format=<protocol> Input format: RFC-5424, RFC-3164 or GELF [default:
RFC-3164].
-h, --help Show this help message and exit.
--[no-]ansi Output in ANSI colors [default: true].
--[no-]stdout Output messages to stdout [default: true].
--[no-]tcp Listen on TCP [default: true].
--[no-]udp Listen on UDP [default: true].
-p, --port=<num> Listening port [default: 1514].
--to-gelf=<uri> Forward to Graylog <udp://host:port>.
--to-loki=<url> Forward to Grafana Loki <http://host:port>.
--to-syslog=<uri> Forward to Syslog <udp://host:port> (RFC-5424).
-V, --version Print version information and exit.
```
The default syslog port (514) requires you to run syslogd as root / administrator.
Any port number above 1024 does not require privileges and can be selected with the *-p* or *--port* option.
### Examples
Listening on the default syslog port:
```
java -jar /path/to/syslogd-x.y.z-all.jar --port 514
```
or, if installed as a *deb* or *rpm* package:
```
/opt/syslogd/bin/syslogd --port 514
```
Forwarding messages on to another log-system on a non-standard port.
```
java -jar /path/to/syslogd-x.y.z-all.jar --to-syslog udp://remotehost:514
```
Forwarding messages to a Graylog server in GELF format.
```
java -jar /path/to/syslogd-x.y.z-all.jar --to-gelf udp://remotehost:12201
```
Forwarding to a Grafana Loki server.
```
java -jar /path/to/syslogd-x.y.z-all.jar --to-loki http://remotehost:3100
```
If you don't want any output locally (only forwarding), you can use the ```--no-stdout``` flag.
## Notes
### IBM AIX and VIO Servers
Syslog messages from AIX (and IBM Power Virtual I/O Servers) can be troublesome with some logging solutions. These can be received with
*syslogd* and then forwarded on to your preferred logging solution.
### Forwarding to Grafana Loki
Forwarding is currently done by making HTTP connections to the Loki API, which works fine for low volume messages, but might cause issues for large volume of messages.
## Development Notes
### Test Grafana Loki
Run Loki and Grafana in local containers to test.
```shell
docker run --rm -d --name=loki -p 3100:3100 grafana/loki
docker run --rm -d --name=grafana --link loki:loki -p 3000:3000 grafana/grafana:7.1.3
```
Please visit [github.com/mnellemann/syslogd](https://github.com/mnellemann/syslogd)

View File

@ -3,9 +3,9 @@ plugins {
id 'groovy' id 'groovy'
id 'application' id 'application'
id 'jacoco' id 'jacoco'
id "com.github.johnrengelman.shadow" version "7.1.2"
id "net.nemerosa.versioning" version "2.15.1" id "net.nemerosa.versioning" version "2.15.1"
id "nebula.ospackage" version "9.1.1" id "com.netflix.nebula.ospackage" version "11.5.0"
id "com.github.johnrengelman.shadow" version "7.1.2"
} }
repositories { repositories {
@ -13,12 +13,12 @@ repositories {
} }
dependencies { dependencies {
annotationProcessor 'info.picocli:picocli-codegen:4.7.0' annotationProcessor 'info.picocli:picocli-codegen:4.7.5'
implementation 'info.picocli:picocli:4.7.0' implementation 'info.picocli:picocli:4.7.5'
implementation 'org.slf4j:slf4j-api:2.0.5' implementation 'org.slf4j:slf4j-api:2.0.9'
implementation 'org.slf4j:slf4j-simple:2.0.5' implementation 'org.slf4j:slf4j-simple:2.0.9'
implementation 'com.fasterxml.jackson.core:jackson-databind:2.14.1' implementation 'com.fasterxml.jackson.core:jackson-databind:2.15.2'
implementation 'com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.14.1' implementation 'com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.15.2'
implementation 'org.apache.commons:commons-collections4:4.4' implementation 'org.apache.commons:commons-collections4:4.4'
testImplementation 'org.spockframework:spock-core:2.3-groovy-3.0' testImplementation 'org.spockframework:spock-core:2.3-groovy-3.0'
@ -38,7 +38,7 @@ test {
} }
jacoco { jacoco {
toolVersion = "0.8.8" toolVersion = "0.8.10"
} }
jacocoTestReport { jacocoTestReport {
@ -71,8 +71,6 @@ jacocoTestCoverageVerification {
} }
check.dependsOn jacocoTestCoverageVerification check.dependsOn jacocoTestCoverageVerification
apply plugin: 'nebula.ospackage'
ospackage { ospackage {
packageName = 'syslogd' packageName = 'syslogd'
release = '1' release = '1'
@ -122,7 +120,7 @@ jar {
} }
} }
tasks.create("packages") { tasks.register("packages") {
group "build" group "build"
dependsOn ":build" dependsOn ":build"
dependsOn ":buildDeb" dependsOn ":buildDeb"

View File

@ -8,7 +8,7 @@ More information about the AIX errlogger is available on the IBM [knowledge cent
### Prepare the local syslog service ### Prepare the local syslog service
Configure the local syslog service to forward messages to our remote [syslogd](https://bitbucket.org/mnellemann/syslogd/) service. Configure the local syslog service to forward messages to our remote [syslogd](https://git.data.coop/nellemann/syslogd/) service.
Create an empty local log file: Create an empty local log file:
@ -39,7 +39,7 @@ refresh -s syslogd
### Forward errlogger to the local syslog ### Forward errlogger to the local syslog
We configure the errloger to forward messages to the local syslog service. We configure the AIX [error logger](https://www.ibm.com/docs/en/aix/7.3?topic=concepts-error-logging-overview) to forward messages to the local syslog service.
Create an odm errnotify logging template file: Create an odm errnotify logging template file:

View File

@ -1,11 +1,11 @@
# Syslogd as a System Service # Syslogd as a system service
## Systemd ## For systemd
Edit the **syslogd.service** and configure required options. To install as a systemd service, copy the [syslogd.service](syslogd.service)
file into */etc/systemd/system/*, edit the file and configure your required options.
To install as a systemd service, copy the **syslogd.service** Enable and start the service:
file into */etc/systemd/system/* and enable the service:
```shell ```shell
systemctl daemon-reload systemctl daemon-reload

View File

@ -1 +1 @@
<mxfile host="drawio-plugin" modified="2022-12-07T07:21:50.025Z" agent="5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.102 Safari/537.36" etag="0NRdR40T7b5zyLZSsQHk" version="20.5.3" type="embed"><diagram id="23iRSUPoRavnBvh4doch" name="Page-1"><mxGraphModel dx="1115" dy="620" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="1169" pageHeight="827" math="0" shadow="0"><root><mxCell id="0"/><mxCell id="1" parent="0"/><mxCell id="10" style="edgeStyle=orthogonalEdgeStyle;orthogonalLoop=1;jettySize=auto;html=1;curved=1;sketch=1;shadow=1;" parent="1" source="2" target="3" edge="1"><mxGeometry relative="1" as="geometry"/></mxCell><mxCell id="2" value="RFC--3164" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#ffe6cc;strokeColor=#d79b00;" parent="1" vertex="1"><mxGeometry x="100" y="40" width="120" height="60" as="geometry"/></mxCell><mxCell id="13" style="edgeStyle=orthogonalEdgeStyle;curved=1;sketch=1;orthogonalLoop=1;jettySize=auto;html=1;shadow=1;" parent="1" source="3" target="9" edge="1"><mxGeometry relative="1" as="geometry"/></mxCell><mxCell id="14" style="edgeStyle=orthogonalEdgeStyle;curved=1;sketch=1;orthogonalLoop=1;jettySize=auto;html=1;shadow=1;" parent="1" source="3" target="6" edge="1"><mxGeometry relative="1" as="geometry"/></mxCell><mxCell id="15" style="edgeStyle=orthogonalEdgeStyle;curved=1;sketch=1;orthogonalLoop=1;jettySize=auto;html=1;shadow=1;" parent="1" source="3" target="7" edge="1"><mxGeometry relative="1" as="geometry"/></mxCell><mxCell id="16" style="edgeStyle=orthogonalEdgeStyle;curved=1;sketch=1;orthogonalLoop=1;jettySize=auto;html=1;shadow=1;" parent="1" source="3" target="8" edge="1"><mxGeometry relative="1" as="geometry"/></mxCell><mxCell id="3" value="syslogd" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;sketch=1;rounded=1;shadow=1;fillColor=#dae8fc;strokeColor=#6c8ebf;" parent="1" vertex="1"><mxGeometry x="280" y="130" width="120" height="60" as="geometry"/></mxCell><mxCell id="11" style="edgeStyle=orthogonalEdgeStyle;curved=1;sketch=1;orthogonalLoop=1;jettySize=auto;html=1;shadow=1;" parent="1" source="4" target="3" edge="1"><mxGeometry relative="1" as="geometry"/></mxCell><mxCell id="4" value="RFC--5424" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#fff2cc;strokeColor=#d6b656;" parent="1" vertex="1"><mxGeometry x="20" y="130" width="120" height="60" as="geometry"/></mxCell><mxCell id="12" style="edgeStyle=orthogonalEdgeStyle;curved=1;sketch=1;orthogonalLoop=1;jettySize=auto;html=1;shadow=1;" parent="1" source="5" target="3" edge="1"><mxGeometry relative="1" as="geometry"/></mxCell><mxCell id="5" value="GELF" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#f8cecc;strokeColor=#b85450;" parent="1" vertex="1"><mxGeometry x="130" y="223.5" width="120" height="60" as="geometry"/></mxCell><mxCell id="6" value="Syslog&lt;br&gt;RFC-5424" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#d5e8d4;strokeColor=#82b366;" parent="1" vertex="1"><mxGeometry x="570" y="80" width="120" height="60" as="geometry"/></mxCell><mxCell id="7" value="Grafana Loki" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#e1d5e7;strokeColor=#9673a6;" parent="1" vertex="1"><mxGeometry x="550" y="170" width="120" height="60" as="geometry"/></mxCell><mxCell id="8" value="Graylog" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#fff2cc;strokeColor=#d6b656;" parent="1" vertex="1"><mxGeometry x="420" y="250" width="120" height="60" as="geometry"/></mxCell><mxCell id="9" value="Standard&lt;br&gt;Output" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#f5f5f5;strokeColor=#666666;fontColor=#333333;" parent="1" vertex="1"><mxGeometry x="410" y="30" width="120" height="60" as="geometry"/></mxCell></root></mxGraphModel></diagram></mxfile> <mxfile host="drawio-plugin" modified="2023-02-05T13:04:08.556Z" agent="5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.102 Safari/537.36" etag="__kq9uG-1g-sjP8t85Xj" version="20.5.3" type="embed"><diagram id="23iRSUPoRavnBvh4doch" name="Page-1"><mxGraphModel dx="809" dy="749" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="1169" pageHeight="827" math="0" shadow="0"><root><mxCell id="0"/><mxCell id="1" parent="0"/><mxCell id="10" style="edgeStyle=orthogonalEdgeStyle;orthogonalLoop=1;jettySize=auto;html=1;curved=1;sketch=1;shadow=1;" parent="1" source="2" target="3" edge="1"><mxGeometry relative="1" as="geometry"/></mxCell><mxCell id="2" value="Syslog&lt;br&gt;RFC--3164" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#ffe6cc;strokeColor=#d79b00;" parent="1" vertex="1"><mxGeometry x="50" y="90" width="120" height="60" as="geometry"/></mxCell><mxCell id="13" style="edgeStyle=orthogonalEdgeStyle;curved=1;sketch=1;orthogonalLoop=1;jettySize=auto;html=1;shadow=1;" parent="1" source="3" target="9" edge="1"><mxGeometry relative="1" as="geometry"/></mxCell><mxCell id="14" style="edgeStyle=orthogonalEdgeStyle;curved=1;sketch=1;orthogonalLoop=1;jettySize=auto;html=1;shadow=1;" parent="1" source="3" target="6" edge="1"><mxGeometry relative="1" as="geometry"/></mxCell><mxCell id="15" style="edgeStyle=orthogonalEdgeStyle;curved=1;sketch=1;orthogonalLoop=1;jettySize=auto;html=1;shadow=1;" parent="1" source="3" target="7" edge="1"><mxGeometry relative="1" as="geometry"/></mxCell><mxCell id="16" style="edgeStyle=orthogonalEdgeStyle;curved=1;sketch=1;orthogonalLoop=1;jettySize=auto;html=1;shadow=1;" parent="1" source="3" target="8" edge="1"><mxGeometry relative="1" as="geometry"/></mxCell><mxCell id="3" value="syslogd" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;sketch=1;rounded=1;shadow=1;fillColor=#dae8fc;strokeColor=#6c8ebf;" parent="1" vertex="1"><mxGeometry x="280" y="130" width="120" height="60" as="geometry"/></mxCell><mxCell id="11" style="edgeStyle=orthogonalEdgeStyle;curved=1;sketch=1;orthogonalLoop=1;jettySize=auto;html=1;shadow=1;" parent="1" source="4" target="3" edge="1"><mxGeometry relative="1" as="geometry"/></mxCell><mxCell id="4" value="Syslog&lt;br&gt;RFC--5424" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#fff2cc;strokeColor=#d6b656;" parent="1" vertex="1"><mxGeometry x="40" y="180" width="120" height="60" as="geometry"/></mxCell><mxCell id="12" style="edgeStyle=orthogonalEdgeStyle;curved=1;sketch=1;orthogonalLoop=1;jettySize=auto;html=1;shadow=1;" parent="1" source="5" target="3" edge="1"><mxGeometry relative="1" as="geometry"/></mxCell><mxCell id="5" value="GELF" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#f8cecc;strokeColor=#b85450;" parent="1" vertex="1"><mxGeometry x="120" y="260" width="120" height="60" as="geometry"/></mxCell><mxCell id="6" value="Syslog&lt;br&gt;RFC-5424" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#d5e8d4;strokeColor=#82b366;" parent="1" vertex="1"><mxGeometry x="570" y="80" width="120" height="60" as="geometry"/></mxCell><mxCell id="7" value="Grafana Loki" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#e1d5e7;strokeColor=#9673a6;" parent="1" vertex="1"><mxGeometry x="550" y="170" width="120" height="60" as="geometry"/></mxCell><mxCell id="8" value="Graylog" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#fff2cc;strokeColor=#d6b656;" parent="1" vertex="1"><mxGeometry x="420" y="250" width="120" height="60" as="geometry"/></mxCell><mxCell id="9" value="Standard&lt;br&gt;Output" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#f5f5f5;strokeColor=#666666;fontColor=#333333;" parent="1" vertex="1"><mxGeometry x="410" y="30" width="120" height="60" as="geometry"/></mxCell><mxCell id="17" value="Standard&lt;br&gt;Input" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#f5f5f5;strokeColor=#666666;fontColor=#333333;" vertex="1" parent="1"><mxGeometry x="180" y="30" width="90" height="60" as="geometry"/></mxCell><mxCell id="19" style="edgeStyle=orthogonalEdgeStyle;orthogonalLoop=1;jettySize=auto;html=1;curved=1;sketch=1;shadow=1;" edge="1" parent="1" source="17" target="3"><mxGeometry relative="1" as="geometry"><mxPoint x="190" y="110" as="sourcePoint"/><mxPoint x="300" y="170" as="targetPoint"/></mxGeometry></mxCell></root></mxGraphModel></diagram></mxfile>

Binary file not shown.

Before

Width:  |  Height:  |  Size: 172 KiB

After

Width:  |  Height:  |  Size: 96 KiB

View File

@ -4,7 +4,7 @@ Description=Syslog Director
[Service] [Service]
TimeoutStartSec=0 TimeoutStartSec=0
Restart=always Restart=always
ExecStart=/opt/syslogd/bin/syslogd --port 514 --no-stdout --syslog=udp://localhost:1514 ExecStart=/opt/syslogd/bin/syslogd --port 514 --no-ansi
[Install] [Install]
WantedBy=default.target WantedBy=default.target

View File

@ -1,5 +1,5 @@
id = syslogd id = syslogd
name = syslogd name = syslogd
group = biz.nellemann.syslogd group = biz.nellemann.syslogd
version = 1.3.1 version = 1.3.5
description = "Syslog Director" description = "Syslog Director"

Binary file not shown.

View File

@ -1,5 +1,6 @@
distributionBase=GRADLE_USER_HOME distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-7.4-bin.zip distributionUrl=https\://services.gradle.org/distributions/gradle-7.6.2-bin.zip
networkTimeout=10000
zipStoreBase=GRADLE_USER_HOME zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists zipStorePath=wrapper/dists

18
gradlew vendored
View File

@ -55,7 +55,7 @@
# Darwin, MinGW, and NonStop. # Darwin, MinGW, and NonStop.
# #
# (3) This script is generated from the Groovy template # (3) This script is generated from the Groovy template
# https://github.com/gradle/gradle/blob/master/subprojects/plugins/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt # https://github.com/gradle/gradle/blob/HEAD/subprojects/plugins/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt
# within the Gradle project. # within the Gradle project.
# #
# You can find Gradle at https://github.com/gradle/gradle/. # You can find Gradle at https://github.com/gradle/gradle/.
@ -80,10 +80,10 @@ do
esac esac
done done
APP_HOME=$( cd "${APP_HOME:-./}" && pwd -P ) || exit # This is normally unused
# shellcheck disable=SC2034
APP_NAME="Gradle"
APP_BASE_NAME=${0##*/} APP_BASE_NAME=${0##*/}
APP_HOME=$( cd "${APP_HOME:-./}" && pwd -P ) || exit
# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. # Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"' DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
@ -143,12 +143,16 @@ fi
if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then
case $MAX_FD in #( case $MAX_FD in #(
max*) max*)
# In POSIX sh, ulimit -H is undefined. That's why the result is checked to see if it worked.
# shellcheck disable=SC3045
MAX_FD=$( ulimit -H -n ) || MAX_FD=$( ulimit -H -n ) ||
warn "Could not query maximum file descriptor limit" warn "Could not query maximum file descriptor limit"
esac esac
case $MAX_FD in #( case $MAX_FD in #(
'' | soft) :;; #( '' | soft) :;; #(
*) *)
# In POSIX sh, ulimit -n is undefined. That's why the result is checked to see if it worked.
# shellcheck disable=SC3045
ulimit -n "$MAX_FD" || ulimit -n "$MAX_FD" ||
warn "Could not set maximum file descriptor limit to $MAX_FD" warn "Could not set maximum file descriptor limit to $MAX_FD"
esac esac
@ -205,6 +209,12 @@ set -- \
org.gradle.wrapper.GradleWrapperMain \ org.gradle.wrapper.GradleWrapperMain \
"$@" "$@"
# Stop when "xargs" is not available.
if ! command -v xargs >/dev/null 2>&1
then
die "xargs is not available"
fi
# Use "xargs" to parse quoted args. # Use "xargs" to parse quoted args.
# #
# With -n1 it outputs one arg per line, with the quotes and backslashes removed. # With -n1 it outputs one arg per line, with the quotes and backslashes removed.

15
gradlew.bat vendored
View File

@ -14,7 +14,7 @@
@rem limitations under the License. @rem limitations under the License.
@rem @rem
@if "%DEBUG%" == "" @echo off @if "%DEBUG%"=="" @echo off
@rem ########################################################################## @rem ##########################################################################
@rem @rem
@rem Gradle startup script for Windows @rem Gradle startup script for Windows
@ -25,7 +25,8 @@
if "%OS%"=="Windows_NT" setlocal if "%OS%"=="Windows_NT" setlocal
set DIRNAME=%~dp0 set DIRNAME=%~dp0
if "%DIRNAME%" == "" set DIRNAME=. if "%DIRNAME%"=="" set DIRNAME=.
@rem This is normally unused
set APP_BASE_NAME=%~n0 set APP_BASE_NAME=%~n0
set APP_HOME=%DIRNAME% set APP_HOME=%DIRNAME%
@ -40,7 +41,7 @@ if defined JAVA_HOME goto findJavaFromJavaHome
set JAVA_EXE=java.exe set JAVA_EXE=java.exe
%JAVA_EXE% -version >NUL 2>&1 %JAVA_EXE% -version >NUL 2>&1
if "%ERRORLEVEL%" == "0" goto execute if %ERRORLEVEL% equ 0 goto execute
echo. echo.
echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
@ -75,13 +76,15 @@ set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar
:end :end
@rem End local scope for the variables with windows NT shell @rem End local scope for the variables with windows NT shell
if "%ERRORLEVEL%"=="0" goto mainEnd if %ERRORLEVEL% equ 0 goto mainEnd
:fail :fail
rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of
rem the _cmd.exe /c_ return code! rem the _cmd.exe /c_ return code!
if not "" == "%GRADLE_EXIT_CONSOLE%" exit 1 set EXIT_CODE=%ERRORLEVEL%
exit /b 1 if %EXIT_CODE% equ 0 set EXIT_CODE=1
if not ""=="%GRADLE_EXIT_CONSOLE%" exit %EXIT_CODE%
exit /b %EXIT_CODE%
:mainEnd :mainEnd
if "%OS%"=="Windows_NT" endlocal if "%OS%"=="Windows_NT" endlocal

View File

@ -1,10 +1 @@
/*
* This file was generated by the Gradle 'init' task.
*
* The settings file is used to specify which projects to include in your build.
*
* Detailed information about configuring a multi-project build in Gradle can be found
* in the user manual at https://docs.gradle.org/6.6.1/userguide/multi_project_builds.html
*/
rootProject.name = 'syslogd' rootProject.name = 'syslogd'

View File

@ -15,16 +15,6 @@
*/ */
package biz.nellemann.syslogd; package biz.nellemann.syslogd;
import biz.nellemann.syslogd.msg.SyslogMessage;
import biz.nellemann.syslogd.net.*;
import biz.nellemann.syslogd.parser.GelfParser;
import biz.nellemann.syslogd.parser.SyslogParser;
import biz.nellemann.syslogd.parser.SyslogParserRfc3164;
import biz.nellemann.syslogd.parser.SyslogParserRfc5424;
import picocli.CommandLine;
import picocli.CommandLine.Command;
import java.io.IOException; import java.io.IOException;
import java.net.InetSocketAddress; import java.net.InetSocketAddress;
import java.net.URI; import java.net.URI;
@ -34,6 +24,19 @@ import java.util.List;
import java.util.Locale; import java.util.Locale;
import java.util.concurrent.Callable; import java.util.concurrent.Callable;
import biz.nellemann.syslogd.msg.SyslogMessage;
import biz.nellemann.syslogd.net.GelfClient;
import biz.nellemann.syslogd.net.LokiClient;
import biz.nellemann.syslogd.net.TcpServer;
import biz.nellemann.syslogd.net.UdpClient;
import biz.nellemann.syslogd.net.UdpServer;
import biz.nellemann.syslogd.parser.GelfParser;
import biz.nellemann.syslogd.parser.SyslogParser;
import biz.nellemann.syslogd.parser.SyslogParserRfc3164;
import biz.nellemann.syslogd.parser.SyslogParserRfc5424;
import picocli.CommandLine;
import picocli.CommandLine.Command;
@Command(name = "syslogd", @Command(name = "syslogd",
mixinStandardHelpOptions = true, mixinStandardHelpOptions = true,
versionProvider = biz.nellemann.syslogd.VersionProvider.class) versionProvider = biz.nellemann.syslogd.VersionProvider.class)
@ -58,6 +61,9 @@ public class Application implements Callable<Integer>, LogReceiveListener {
@CommandLine.Option(names = "--no-stdout", negatable = true, description = "Output messages to stdout [default: true].", defaultValue = "true") @CommandLine.Option(names = "--no-stdout", negatable = true, description = "Output messages to stdout [default: true].", defaultValue = "true")
private boolean stdout; private boolean stdout;
@CommandLine.Option(names = "--no-stdin", negatable = true, description = "Forward messages from stdin [default: true].", defaultValue = "true")
private boolean stdin;
@CommandLine.Option(names = {"-f", "--format"}, description = "Input format: RFC-5424, RFC-3164 or GELF [default: RFC-3164].", defaultValue = "RFC-3164") @CommandLine.Option(names = {"-f", "--format"}, description = "Input format: RFC-5424, RFC-3164 or GELF [default: RFC-3164].", defaultValue = "RFC-3164")
private String protocol; private String protocol;
@ -77,7 +83,6 @@ public class Application implements Callable<Integer>, LogReceiveListener {
@Override @Override
public Integer call() throws IOException { public Integer call() throws IOException {
if(enableDebug) { if(enableDebug) {
System.setProperty("org.slf4j.simpleLogger.defaultLogLevel", "DEBUG"); System.setProperty("org.slf4j.simpleLogger.defaultLogLevel", "DEBUG");
} }
@ -115,6 +120,12 @@ public class Application implements Callable<Integer>, LogReceiveListener {
t.start(); t.start();
} }
if(stdin) {
InputReader inputReader = new InputReader(System.in, protocol);
inputReader.addEventListener(this);
inputReader.start();
}
if(udpServer) { if(udpServer) {
UdpServer udpServer = new UdpServer(port); UdpServer udpServer = new UdpServer(port);
udpServer.addEventListener(this); udpServer.addEventListener(this);
@ -144,7 +155,7 @@ public class Application implements Callable<Integer>, LogReceiveListener {
if(msg != null) { if(msg != null) {
if(logForwardListeners.size() > 0) { if(!logForwardListeners.isEmpty()) {
sendForwardEvent(msg); sendForwardEvent(msg);
} }

View File

@ -0,0 +1,70 @@
package biz.nellemann.syslogd;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.List;
import java.util.Scanner;
import biz.nellemann.syslogd.msg.SyslogMessage;
public class InputReader extends Thread {
private final Scanner input;
private final String protocol;
public InputReader(InputStream inputStream, String protocol) {
input = new Scanner(inputStream);
this.protocol = protocol;
}
@Override
public void run() {
while(input.hasNextLine()) {
SyslogMessage msg = new SyslogMessage(input.nextLine());
msg.hostname = "localhost";
msg.application = "syslogd";
String payload;
if(protocol.equalsIgnoreCase("GELF"))
payload = SyslogPrinter.toGelf(msg);
else if (protocol.equalsIgnoreCase("RFC-5424")) {
payload = SyslogPrinter.toRfc5424(msg);
} else {
payload = SyslogPrinter.toRfc3164(msg);
}
sendEvent(payload);
}
input.close();
}
private synchronized void sendEvent(String text) {
LogReceiveEvent event = new LogReceiveEvent( this, text);
for (LogReceiveListener eventListener : eventListeners) {
eventListener.onLogEvent(event);
}
}
/**
* Event Listener Configuration
*/
protected List<LogReceiveListener> eventListeners = new ArrayList<>();
public synchronized void addEventListener(LogReceiveListener listener ) {
eventListeners.add( listener );
}
public synchronized void addEventListener(List<LogReceiveListener> listeners ) {
eventListeners.addAll(listeners);
}
public synchronized void removeEventListener( LogReceiveListener l ) {
eventListeners.remove( l );
}
}

View File

@ -23,15 +23,13 @@ import java.util.EventObject;
public class LogReceiveEvent extends EventObject { public class LogReceiveEvent extends EventObject {
private static final long serialVersionUID = 1L; private static final long serialVersionUID = 1L;
//private final String message;
private final DatagramPacket packet; private final DatagramPacket packet;
/*
public LogReceiveEvent(final Object source, final String message ) { public LogReceiveEvent(final Object source, final String message ) {
super( source ); super( source );
this.message = message; byte[] bytes = message.getBytes();
this.packet = new DatagramPacket(bytes, bytes.length);
} }
*/
public LogReceiveEvent(final Object source, final DatagramPacket packet) { public LogReceiveEvent(final Object source, final DatagramPacket packet) {
super( source ); super( source );

View File

@ -103,18 +103,18 @@ public class SyslogPrinter {
* @return * @return
*/ */
public static String toGelf(SyslogMessage msg) { public static String toGelf(SyslogMessage msg) {
StringBuilder sb = new StringBuilder("{ \"version\": \"1.1\","); StringBuilder sb = new StringBuilder("{ \"version\": \"1.1\"");
sb.append(String.format("\"host\": \"%s\",", msg.hostname)); sb.append(String.format(", \"host\": \"%s\"", msg.hostname));
sb.append(String.format("\"short_message\": \"%s\",", JsonUtil.encode(msg.message))); sb.append(String.format(", \"short_message\": \"%s\"", JsonUtil.encode(msg.message)));
sb.append(String.format("\"full_message\": \"%s\",", msg.structuredData)); sb.append(String.format(", \"full_message\": \"%s\"", JsonUtil.encode(msg.structuredData)));
sb.append(String.format("\"timestamp\": %d,", msg.timestamp.getEpochSecond())); sb.append(String.format(", \"timestamp\": %d", msg.timestamp.getEpochSecond()));
sb.append(String.format("\"level\": %d,", msg.severity.toNumber())); sb.append(String.format(", \"level\": %d", msg.severity.toNumber()));
sb.append(String.format("\"_facility\": \"%s\",", msg.facility)); sb.append(String.format(", \"_facility\": \"%s\"", msg.facility));
sb.append(String.format("\"_severity\": \"%s\",", msg.severity)); sb.append(String.format(", \"_severity\": \"%s\"", msg.severity));
sb.append(String.format("\"_application\": \"%s\",", msg.application)); sb.append(String.format(", \"_application\": \"%s\"", msg.application));
if(msg.processId != null) { sb.append(String.format("\"_process-id\": \"%s\",", msg.processId)); } if(msg.processId != null) { sb.append(String.format(", \"_process-id\": \"%s\"", msg.processId)); }
if(msg.messageId != null) { sb.append(String.format("\"_message-id\": \"%s\",", msg.messageId)); } if(msg.messageId != null) { sb.append(String.format(", \"_message-id\": \"%s\"", msg.messageId)); }
if(msg.structuredData != null) { sb.append(String.format("\"_structured-data\": \"%s\",", msg.structuredData)); } if(msg.structuredData != null) { sb.append(String.format(", \"_structured-data\": \"%s\"", JsonUtil.encode(msg.structuredData))); }
sb.append("}"); sb.append("}");
return sb.toString(); return sb.toString();
} }

View File

@ -36,7 +36,7 @@ public class SyslogMessage {
// The TIMESTAMP field is a formalized timestamp derived from [RFC3339]. // The TIMESTAMP field is a formalized timestamp derived from [RFC3339].
@JsonProperty("timestamp") // 1670357783.694 - in GELF: seconds since UNIX epoch with optional decimal places for milliseconds @JsonProperty("timestamp") // 1670357783.694 - in GELF: seconds since UNIX epoch with optional decimal places for milliseconds
public Instant timestamp; public Instant timestamp = Instant.now();
// The HOSTNAME field identifies the machine that originally sent the syslog message. // The HOSTNAME field identifies the machine that originally sent the syslog message.
@JsonProperty("host") @JsonProperty("host")

View File

@ -1,13 +1,14 @@
package biz.nellemann.syslogd.net; package biz.nellemann.syslogd.net;
import biz.nellemann.syslogd.LogForwardEvent;
import biz.nellemann.syslogd.SyslogPrinter;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.net.InetSocketAddress; import java.net.InetSocketAddress;
import java.net.SocketException; import java.net.SocketException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import biz.nellemann.syslogd.LogForwardEvent;
import biz.nellemann.syslogd.SyslogPrinter;
public class GelfClient extends UdpClient { public class GelfClient extends UdpClient {
private final static Logger log = LoggerFactory.getLogger(GelfClient.class); private final static Logger log = LoggerFactory.getLogger(GelfClient.class);

View File

@ -15,19 +15,22 @@
*/ */
package biz.nellemann.syslogd.net; package biz.nellemann.syslogd.net;
import biz.nellemann.syslogd.LogForwardEvent;
import biz.nellemann.syslogd.LogForwardListener;
import biz.nellemann.syslogd.SyslogPrinter;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException; import java.io.IOException;
import java.io.InputStream; import java.io.InputStream;
import java.io.OutputStream; import java.io.OutputStream;
import java.net.*; import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
import java.nio.charset.StandardCharsets; import java.nio.charset.StandardCharsets;
import java.util.concurrent.ArrayBlockingQueue; import java.util.concurrent.ArrayBlockingQueue;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import biz.nellemann.syslogd.LogForwardEvent;
import biz.nellemann.syslogd.LogForwardListener;
import biz.nellemann.syslogd.SyslogPrinter;
public class LokiClient implements LogForwardListener, Runnable { public class LokiClient implements LogForwardListener, Runnable {
private final static Logger log = LoggerFactory.getLogger(LokiClient.class); private final static Logger log = LoggerFactory.getLogger(LokiClient.class);
@ -84,7 +87,7 @@ public class LokiClient implements LogForwardListener, Runnable {
while (true) { while (true) {
try { try {
send(blockingQueue.take()); send(blockingQueue.take());
} catch (Exception e) { } catch (MalformedURLException | InterruptedException e) {
log.warn(e.getMessage()); log.warn(e.getMessage());
} }
} }

View File

@ -26,10 +26,16 @@ import java.net.ServerSocket;
import java.net.Socket; import java.net.Socket;
import java.nio.charset.StandardCharsets; import java.nio.charset.StandardCharsets;
import java.util.List; import java.util.List;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.ArrayList; import java.util.ArrayList;
public class TcpServer { public class TcpServer {
private final static Logger log = LoggerFactory.getLogger(TcpServer.class);
private final int port; private final int port;
private ServerSocket serverSocket; private ServerSocket serverSocket;
@ -91,14 +97,14 @@ public class TcpServer {
sendEvent(inputLine); sendEvent(inputLine);
} }
} catch (IOException e) { } catch (IOException e) {
System.err.printf("TcpServer - error: %s\n", e.getMessage()); log.warn("run() - read error: {}", e.getMessage());
} }
try { try {
in.close(); in.close();
clientSocket.close(); clientSocket.close();
} catch (IOException e) { } catch (IOException e) {
System.err.printf("TcpServer - error: %s\n", e.getMessage()); log.warn("run() - close error: {}", e.getMessage());
} }
} }

View File

@ -15,18 +15,22 @@
*/ */
package biz.nellemann.syslogd.net; package biz.nellemann.syslogd.net;
import biz.nellemann.syslogd.LogReceiveEvent;
import biz.nellemann.syslogd.LogReceiveListener;
import java.io.IOException; import java.io.IOException;
import java.net.DatagramPacket; import java.net.DatagramPacket;
import java.net.DatagramSocket; import java.net.DatagramSocket;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.List; import java.util.List;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import biz.nellemann.syslogd.LogReceiveEvent;
import biz.nellemann.syslogd.LogReceiveListener;
public class UdpServer extends Thread { public class UdpServer extends Thread {
private final static Logger log = LoggerFactory.getLogger(UdpServer.class);
protected DatagramSocket socket; protected DatagramSocket socket;
protected boolean listen = true; protected boolean listen = true;
@ -34,6 +38,7 @@ public class UdpServer extends Thread {
socket = new DatagramSocket(port); socket = new DatagramSocket(port);
} }
@Override
public void run() { public void run() {
byte[] buf = new byte[8192]; byte[] buf = new byte[8192];
@ -44,8 +49,8 @@ public class UdpServer extends Thread {
socket.receive(packet); socket.receive(packet);
//String packetData = new String(packet.getData(), packet.getOffset(), packet.getLength(), StandardCharsets.UTF_8); //String packetData = new String(packet.getData(), packet.getOffset(), packet.getLength(), StandardCharsets.UTF_8);
sendEvent(packet); sendEvent(packet);
} catch (Exception e) { } catch (IOException e) {
e.printStackTrace(); log.error("run() - error: {}", e.getMessage());
listen = false; listen = false;
} }
} }

View File

@ -1,17 +1,19 @@
package biz.nellemann.syslogd.parser; package biz.nellemann.syslogd.parser;
import biz.nellemann.syslogd.msg.SyslogMessage;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule;
import org.apache.commons.collections4.map.PassiveExpiringMap;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.time.Instant; import java.time.Instant;
import java.util.Arrays; import java.util.Arrays;
import java.util.TreeMap; import java.util.TreeMap;
import org.apache.commons.collections4.map.PassiveExpiringMap;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule;
import biz.nellemann.syslogd.msg.SyslogMessage;
/* /*
For more information about the GELF format, visit: https://go2docs.graylog.org/5-0/getting_in_log_data/gelf.html For more information about the GELF format, visit: https://go2docs.graylog.org/5-0/getting_in_log_data/gelf.html
*/ */
@ -81,11 +83,12 @@ public class GelfParser extends SyslogParser {
@Override @Override
public SyslogMessage parse(String input) { public SyslogMessage parse(String input) {
if(!input.startsWith("{")) return null; // Avoid trying to parse non-JSON content
SyslogMessage message = null; SyslogMessage message = null;
try { try {
message = objectMapper.readValue(input, SyslogMessage.class); message = objectMapper.readValue(input, SyslogMessage.class);
} catch (JsonProcessingException e) { } catch (JsonProcessingException e) {
log.warn("parse() - error: {}", e.getMessage()); log.debug("parse() - error: {}", e.getMessage());
} }
return message; return message;
} }

View File

@ -8,6 +8,10 @@ public class JsonUtil {
public static String encode(String input) { public static String encode(String input) {
if(input == null) {
return "";
}
StringBuilder output = new StringBuilder(); StringBuilder output = new StringBuilder();
for (int i = 0; i < input.length(); i++) { for (int i = 0; i < input.length(); i++) {

View File

@ -15,16 +15,16 @@
*/ */
package biz.nellemann.syslogd.parser; package biz.nellemann.syslogd.parser;
import biz.nellemann.syslogd.msg.SyslogMessage;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.UnsupportedEncodingException;
import java.nio.charset.StandardCharsets; import java.nio.charset.StandardCharsets;
import java.time.Instant; import java.time.Instant;
import java.util.zip.DataFormatException; import java.util.zip.DataFormatException;
import java.util.zip.Inflater; import java.util.zip.Inflater;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import biz.nellemann.syslogd.msg.SyslogMessage;
public abstract class SyslogParser { public abstract class SyslogParser {
private final static Logger log = LoggerFactory.getLogger(SyslogParser.class); private final static Logger log = LoggerFactory.getLogger(SyslogParser.class);
@ -74,7 +74,7 @@ public abstract class SyslogParser {
Inflater decompressor = new Inflater(); Inflater decompressor = new Inflater();
decompressor.setInput(data, 0, data.length); decompressor.setInput(data, 0, data.length);
//byte[] result = new byte[data.length * 2]; //byte[] result = new byte[data.length * 2];
int resultLength = decompressor.inflate(result); decompressor.inflate(result);
decompressor.end(); decompressor.end();
// Decode the bytes into a String // Decode the bytes into a String

View File

@ -15,24 +15,26 @@
*/ */
package biz.nellemann.syslogd.parser; package biz.nellemann.syslogd.parser;
import biz.nellemann.syslogd.LogReceiveEvent; import java.time.Instant;
import biz.nellemann.syslogd.msg.Facility; import java.time.OffsetDateTime;
import biz.nellemann.syslogd.msg.Severity; import java.time.ZoneId;
import biz.nellemann.syslogd.msg.SyslogMessage;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.time.*;
import java.time.format.DateTimeFormatter; import java.time.format.DateTimeFormatter;
import java.time.format.DateTimeParseException; import java.time.format.DateTimeParseException;
import java.util.regex.Matcher; import java.util.regex.Matcher;
import java.util.regex.Pattern; import java.util.regex.Pattern;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import biz.nellemann.syslogd.msg.Facility;
import biz.nellemann.syslogd.msg.Severity;
import biz.nellemann.syslogd.msg.SyslogMessage;
public class SyslogParserRfc3164 extends SyslogParser { public class SyslogParserRfc3164 extends SyslogParser {
private final static Logger log = LoggerFactory.getLogger(SyslogParserRfc3164.class); private final static Logger log = LoggerFactory.getLogger(SyslogParserRfc3164.class);
private final Pattern pattern = Pattern.compile("^<(\\d{1,3})>(\\D{3}\\s+\\d{1,2} \\d{2}:\\d{2}:\\d{2})\\s+(Message forwarded from \\S+:|\\S+)\\s+([^\\s:]+):?\\s+(.*)", Pattern.CASE_INSENSITIVE); private final Pattern pattern = Pattern.compile("^<(\\d{1,3})>(\\D{3}\\s+\\d{1,2} \\d{2}:\\d{2}:\\d{2})\\s+(Message forwarded from \\S+:|\\S+:?)\\s+([^\\s:]+):?\\s+(.*)", Pattern.CASE_INSENSITIVE);
private final DateTimeFormatter dateTimeFormatter = DateTimeFormatter.ofPattern("yyyy MMM [ ]d HH:mm:ss").withZone(ZoneId.systemDefault()); private final DateTimeFormatter dateTimeFormatter = DateTimeFormatter.ofPattern("yyyy MMM [ ]d HH:mm:ss").withZone(ZoneId.systemDefault());
/** /**
@ -90,6 +92,7 @@ public class SyslogParserRfc3164 extends SyslogParser {
* @param dateString * @param dateString
* @return * @return
*/ */
@Override
public Instant parseTimestamp(String dateString) { public Instant parseTimestamp(String dateString) {
// We need to add current year to parse date correctly // We need to add current year to parse date correctly

View File

@ -15,26 +15,26 @@
*/ */
package biz.nellemann.syslogd.parser; package biz.nellemann.syslogd.parser;
import biz.nellemann.syslogd.LogReceiveEvent;
import biz.nellemann.syslogd.msg.Severity;
import biz.nellemann.syslogd.msg.Facility;
import biz.nellemann.syslogd.msg.SyslogMessage;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.text.ParseException; import java.text.ParseException;
import java.text.SimpleDateFormat; import java.text.SimpleDateFormat;
import java.time.*; import java.time.Instant;
import java.util.Arrays; import java.util.Arrays;
import java.util.List; import java.util.List;
import java.util.regex.Matcher; import java.util.regex.Matcher;
import java.util.regex.Pattern; import java.util.regex.Pattern;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import biz.nellemann.syslogd.msg.Facility;
import biz.nellemann.syslogd.msg.Severity;
import biz.nellemann.syslogd.msg.SyslogMessage;
public class SyslogParserRfc5424 extends SyslogParser { public class SyslogParserRfc5424 extends SyslogParser {
private final static Logger log = LoggerFactory.getLogger(SyslogParserRfc5424.class); private final static Logger log = LoggerFactory.getLogger(SyslogParserRfc5424.class);
private final Pattern pattern = Pattern.compile("^<(\\d{1,3})>(\\d+)\\s+(\\S+)\\s+(\\S+)\\s+(\\S+)\\s+(\\S+)\\s+(\\S+)\\s+(\\[.*\\]|-)\\s+(\\S+)", Pattern.CASE_INSENSITIVE); private final Pattern pattern = Pattern.compile("^<(\\d{1,3})>(\\d+)\\s+(\\S+)\\s+(\\S+)\\s+(\\S+)\\s+(\\S+)\\s+(\\S+)\\s+(\\[.*\\]|-)\\s+(.*)", Pattern.CASE_INSENSITIVE);
/** /**
* Parses [rfc5424](https://tools.ietf.org/html/rfc5424) syslog messages. * Parses [rfc5424](https://tools.ietf.org/html/rfc5424) syslog messages.
@ -99,6 +99,7 @@ public class SyslogParserRfc5424 extends SyslogParser {
* @param dateString * @param dateString
* @return * @return
*/ */
@Override
public Instant parseTimestamp(String dateString) { public Instant parseTimestamp(String dateString) {
/* /*

View File

@ -85,4 +85,16 @@ class GelfParserTest extends Specification {
} }
void "junk GET request"() {
setup:
def input = 'GET /'
when:
SyslogMessage msg = syslogParser.parse(input)
then:
msg == null
}
} }

View File

@ -50,13 +50,13 @@ class SyslogParserRfc3164Test extends Specification {
void "test rfc3164 normal message"() { void "test rfc3164 normal message"() {
setup: setup:
def input = "<13>Sep 23 08:53:28 xps13 mark: adfdfdf3432434" def input = "<13>Sep 23 08:53:28 xps13 mark: adfdfdf3432434 abcdefghijklmnopqrstuvwxyz"
when: when:
SyslogMessage msg = syslogParser.parse(input) SyslogMessage msg = syslogParser.parse(input)
then: then:
msg.message == "adfdfdf3432434" msg.message == "adfdfdf3432434 abcdefghijklmnopqrstuvwxyz"
msg.hostname == "xps13" msg.hostname == "xps13"
msg.application == "mark" msg.application == "mark"
} }

View File

@ -40,6 +40,7 @@ class SyslogParserRfc5424Test extends Specification {
msg.application == "su" msg.application == "su"
msg.messageId == "ID47" msg.messageId == "ID47"
msg.processId == null msg.processId == null
msg.message == "BOM'su root' failed for lonvick on /dev/pts/8"
} }
void "test rfc5424 example2 message"() { void "test rfc5424 example2 message"() {

View File

@ -10,10 +10,23 @@ class SyslogPrinterTest extends Specification {
void setup() { void setup() {
} }
void "to plain"() {
setup:
SyslogParser syslogParser = new SyslogParserRfc5424();
String input = '<13>1 2020-09-23T08:57:30.950699+02:00 xps13 mark - - [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"] adfdfdf3432434565656 abcdefghijklmnopqrstuvwxyz'
SyslogMessage msg = syslogParser.parse(input)
when:
String output = SyslogPrinter.toString(msg)
then:
output.endsWith("abcdefghijklmnopqrstuvwxyz")
}
void "test toGelf"() { void "test toGelf"() {
setup: setup:
SyslogParser syslogParser = new SyslogParserRfc5424(); SyslogParser syslogParser = new SyslogParserRfc5424();
String input = '<13>1 2020-09-23T08:57:30.950699+02:00 xps13 mark - - [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"] adfdfdf3432434565656' String input = '<13>1 2020-09-23T08:57:30.950699+02:00 xps13 mark - - [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"] adfdfdf3432434565656 abcdefghijklmnopqrstuvwxyz'
SyslogMessage msg = syslogParser.parse(input) SyslogMessage msg = syslogParser.parse(input)
when: when:
@ -26,14 +39,14 @@ class SyslogPrinterTest extends Specification {
void "test toLoki"() { void "test toLoki"() {
setup: setup:
SyslogParser syslogParser = new SyslogParserRfc5424(); SyslogParser syslogParser = new SyslogParserRfc5424();
String input = '<13>1 2020-09-23T08:57:30.950699+02:00 xps13 mark - - [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"] adfdfdf3432434565656' String input = '<13>1 2020-09-23T08:57:30.950699+02:00 xps13 mark - - [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"] adfdfdf3432434565656 abcdefghijklmnopqrstuvwxyz'
SyslogMessage msg = syslogParser.parse(input) SyslogMessage msg = syslogParser.parse(input)
when: when:
String output = SyslogPrinter.toLoki(msg) String output = SyslogPrinter.toLoki(msg)
then: then:
output == '{ "streams": [ { "stream": { "hostname": "xps13", "facility": "user", "level": "notice", "application": "mark"}, "values": [ [ "1600845200000000000", "[user.notice] xps13 mark adfdfdf3432434565656" ] ] } ] }' output == '{ "streams": [ { "stream": { "hostname": "xps13", "facility": "user", "level": "notice", "application": "mark"}, "values": [ [ "1600845200000000000", "[user.notice] xps13 mark adfdfdf3432434565656 abcdefghijklmnopqrstuvwxyz" ] ] } ] }'
} }
} }