Compare commits

...

64 Commits
v1.0.4 ... main

Author SHA1 Message Date
Mark Nellemann 3ef961e44b Update README.md 2024-05-17 07:53:25 +00:00
Mark Nellemann bd5e2634d5 Merge pull request 'Depedency updates and cleanup.' (#1) from updates into main
continuous-integration/drone/push Build is passing Details
continuous-integration/drone/tag Build is passing Details
Reviewed-on: #1
2023-11-13 12:48:20 +00:00
Mark Nellemann 1acdd6a93d Cleanup.
continuous-integration/drone/push Build is passing Details
continuous-integration/drone/pr Build is passing Details
2023-11-13 13:46:11 +01:00
Mark Nellemann d39837861f Update dependencies and gradle. 2023-10-02 14:25:33 +02:00
Mark Nellemann 422f1fbb71 Read from stdin and forward.
continuous-integration/drone/push Build is passing Details
continuous-integration/drone/tag Build is passing Details
2023-02-05 14:38:54 +01:00
Mark Nellemann d3589faf9e Fix syslog message parsing error resulting in messages being cut at first whitespace.
continuous-integration/drone/push Build is passing Details
continuous-integration/drone/tag Build is passing Details
2023-01-25 08:47:31 +01:00
Mark Nellemann c47f682c34 Improve robustness of GELF forwarding and parsing.
continuous-integration/drone/push Build is passing Details
continuous-integration/drone/tag Build is passing Details
2023-01-22 11:44:03 +01:00
Mark Nellemann b291f87693 Update links and gradle.
continuous-integration/drone/push Build is passing Details
2023-01-06 08:06:52 +01:00
Mark Nellemann 3e7ba2f46e Cleanup and gradle update.
continuous-integration/drone/push Build is passing Details
2023-01-03 12:49:21 +01:00
Mark Nellemann a0ad98bf52 Update 'README.md'
continuous-integration/drone/push Build is passing Details
2023-01-03 08:24:18 +00:00
Mark Nellemann 9fb810c9d2 Drone CI build pipelines
continuous-integration/drone/tag Build is passing Details
2023-01-03 09:04:08 +01:00
Mark Nellemann 8895b2b110 Update README with links to other related projects. 2022-12-17 10:35:55 +01:00
Mark Nellemann 99466b0037 Bump version. 2022-12-14 08:19:11 +01:00
Mark Nellemann 2284b5eecb Merged in gelf-chunked (pull request #9)
Gelf chunked
2022-12-14 07:15:06 +00:00
Mark Nellemann cd6e15584a Correctly parse chuncked GELF messages. 2022-12-14 08:14:00 +01:00
Mark Nellemann 5104bd0750 Work on chunked GELF messages. 2022-12-13 16:59:28 +01:00
Mark Nellemann 4cc11b0587 Work on detecting chunked messages. 2022-12-13 09:14:21 +01:00
Mark Nellemann 4cb04bd687 Correct example with new syntax. 2022-12-08 13:09:21 +00:00
Mark Nellemann 4a05014dbc Merged in gelf (pull request #8)
Support incoming messages in GELF format (also compressed UDP).
2022-12-08 12:57:48 +00:00
Mark Nellemann 2b52550f87 Support incoming messages in GELF format (also compressed UDP). 2022-12-08 13:55:25 +01:00
Mark Nellemann 3e21ea395e Update dependencies and build container. 2022-12-06 10:15:23 +01:00
Mark Nellemann cf377adecd Update dependencies and build container. 2022-11-02 12:09:49 +01:00
Mark Nellemann 46b70502df Update dependencies and bump version to 1.2.6 2022-02-25 12:49:56 +01:00
Mark Nellemann c38ae5f60b Merged in development (pull request #7)
Development
2021-12-14 21:12:13 +00:00
Mark Nellemann d957e9b438 Update gradle to 7.3.1 2021-12-14 22:08:47 +01:00
Mark Nellemann 326975fcfa Update 3rd party dependencies. 2021-12-03 11:29:55 +01:00
Mark Nellemann 69347764a9 Update HMC syslog ignore filter. 2021-03-29 22:16:15 +02:00
Mark Nellemann e9d9355318 Merged in lokithread (pull request #6)
Move lokiclient into it's own thread, to not block/delay udp forwards.
2021-03-27 17:34:23 +00:00
Mark Nellemann 3b8231cf11 Move lokiclient into it's own thread, to not block/delay udp forwards. 2021-03-27 15:23:34 +01:00
Mark Nellemann 76ee8abe91 Update reference to the Apache 2.0 LICENSE. 2021-03-25 21:21:46 +01:00
Mark Nellemann e92d2b0684 Text cleanup. 2021-03-25 21:13:06 +01:00
Mark Nellemann 7c2762dcff Improve escaping of chars in json output (Loki and GELF). 2021-03-24 11:10:19 +01:00
Mark Nellemann 8e84b9ca5c More logging examples. 2021-03-24 08:05:14 +01:00
Mark Nellemann 49c710ee46 AIX errlogger example. 2021-03-17 22:05:54 +01:00
Mark Nellemann 0cd90c4fef Improve Loki message line. 2021-03-17 21:17:20 +01:00
Mark Nellemann 5e1481e770 Minor version incremented as command line parameters and functionality has changed.
Severity and facility are now in lowercase.
LokiClient http timeouts set more aggressively.
Cleanup various places.
2021-03-17 14:30:37 +01:00
Mark Nellemann bedcaad3b7 README.md edited online with Bitbucket 2021-03-17 10:16:57 +00:00
Mark Nellemann a8195ca7bb Merged in loki (pull request #5)
Loki
2021-03-17 10:11:52 +00:00
Mark Nellemann 93874f1f33 Refactor argument parsing and improve LokiClient send. 2021-03-17 10:45:13 +01:00
Mark Nellemann 883963b033 Work on forwarding syslog messaged to Grafana Loki. 2021-03-16 22:01:11 +01:00
Mark Nellemann 058198003b More narrow output of application in stdout message printer. 2021-02-25 14:28:51 +01:00
Mark Nellemann c7fc3a594c Add more fields to GELF output. 2021-02-05 10:41:04 +01:00
Mark Nellemann 586848e1cc Merged in gelf (pull request #4)
Add option to forward in GELF JSON format.
2021-02-03 11:08:05 +00:00
Mark Nellemann 3313713f36 Add option to forward in GELF JSON format. 2021-02-03 12:00:45 +01:00
Mark Nellemann cf14736d1a Update README to reflect new changes. 2021-01-29 11:18:08 +01:00
Mark Nellemann edfd89a11d Fixes for timestamp parsing. 2021-01-28 09:18:39 +01:00
Mark Nellemann 49b8888032 Hopefully fix failing tests (due to timezone differences) 2021-01-27 20:31:25 +01:00
Mark Nellemann 6fc55f14cb Rewrite test to avoid timezone issues. 2021-01-27 15:26:47 +01:00
Mark Nellemann 0909298bbc Merged in fixes (pull request #3)
Fixes
2021-01-27 14:23:00 +00:00
Mark Nellemann 633ec03bae Refactoring and more tests. 2021-01-27 15:18:46 +01:00
Mark Nellemann 0752c0b6a6 Improve rfc5424 parsing. 2021-01-27 01:29:38 +01:00
Mark Nellemann f9340fb8b2 Fix example to reworked format syntax. 2021-01-26 21:55:18 +01:00
Mark Nellemann dc3fcb0e09 Refactor forwarding logic. 2021-01-26 21:44:23 +01:00
Mark Nellemann 86f645f295 Update instructions and provide systemd service example. 2021-01-26 16:08:37 +01:00
Mark Nellemann c80d8a5ff8 Facility+Severity to PRI value. 2021-01-26 15:52:02 +01:00
Mark Nellemann 5c76a2fafe Merged in forward (pull request #2)
Enable forwarding of received syslog messaged.
2021-01-26 14:26:58 +00:00
Mark Nellemann f2b64383cc Enable forwarding of received syslog messaged. 2021-01-26 15:24:23 +01:00
Mark Nellemann 0e73aa8321 Merged development into master 2021-01-05 15:08:07 +01:00
Mark Nellemann c20137db35 Replace logback w. slf4j-simple to reduce size. 2021-01-05 15:07:26 +01:00
Mark Nellemann 7e8f49edbb Cleanup. 2020-12-15 10:29:54 +01:00
Mark Nellemann 259f90c86c Typos 2020-12-06 13:11:58 +01:00
Mark Nellemann aea5544645 Merged in development (pull request #1)
Cleanup.
2020-12-01 12:45:08 +00:00
Mark Nellemann e1b56b3cc1 Cleanup. 2020-12-01 13:43:36 +01:00
Mark Nellemann 500fd79121 Cleanup. 2020-10-14 08:39:29 +02:00
52 changed files with 2450 additions and 753 deletions

23
.drone.yml Normal file
View File

@ -0,0 +1,23 @@
---
kind: pipeline
name: default
type: docker
steps:
- name: build
image: eclipse-temurin:8-jdk
commands:
- ./gradlew build
- name: publish
image: eclipse-temurin:8-jdk
environment:
AUTH_TOKEN: # Gitea access token ENV variable
from_secret: auth # Name of DroneCI secret exposed above
commands:
- ./gradlew packages
- for file in build/libs/*-all.jar ; do curl --user "${DRONE_REPO_OWNER}:$${AUTH_TOKEN}" --upload-file "$${file}" "https://git.data.coop/api/packages/${DRONE_REPO_OWNER}/generic/${DRONE_REPO_NAME}/${DRONE_TAG}/$(basename $file)" ; done
- for file in build/distributions/*.deb ; do curl --user "${DRONE_REPO_OWNER}:$${AUTH_TOKEN}" --upload-file "$${file}" "https://git.data.coop/api/packages/${DRONE_REPO_OWNER}/generic/${DRONE_REPO_NAME}/${DRONE_TAG}/$(basename $file)" ; done
- for file in build/distributions/*.rpm ; do curl --user "${DRONE_REPO_OWNER}:$${AUTH_TOKEN}" --upload-file "$${file}" "https://git.data.coop/api/packages/${DRONE_REPO_OWNER}/generic/${DRONE_REPO_NAME}/${DRONE_TAG}/$(basename $file)" ; done
when:
event:
- tag

8
.gitignore vendored
View File

@ -1,6 +1,8 @@
# Ignore Gradle project-specific cache directory
.classpath
.settings
.project
.gradle
.vscode
.idea
# Ignore Gradle build output directory
build
bin

View File

@ -1,21 +1,3 @@
# Syslog Daemon
# Repository moved
Basic syslog server written in Java. All received messages are written to *stdout*.
The syslog server is able to listen on UDP and/or TCP and parses syslog messages in either RFC5424 or RFC3164 (BSD) format. The default syslog port (514) requires you to run syslogd as root / administrator. If you do not with to do so, you can choose a port number (with the -p flag) above 1024.
## Usage Instructions
- Install the syslogd package (*.deb* or *.rpm*) from [downloads](https://bitbucket.org/mnellemann/syslogd/downloads/) or compile from source.
- Run *bin/syslogd*, use the *-h* option for help :)
````
Usage: syslogd [-hV] [--[no-]tcp] [--[no-]udp] [--rfc3164] [-p=<port>]
Simple syslog server that prints messages to stdout.
-h, --help Show this help message and exit.
--[no-]tcp Listen on TCP, true by default.
--[no-]udp Listen on UDP, true by default.
-p, --port=<port> Listening port, 514 (privileged) by default.
--rfc3164 Parse RFC3164 syslog message, RFC5424 by default.
-V, --version Print version information and exit.
````
Please visit [github.com/mnellemann/syslogd](https://github.com/mnellemann/syslogd)

View File

@ -1,4 +1,4 @@
image: openjdk:8
image: eclipse-temurin:8-jdk
pipelines:
branches:

View File

@ -2,35 +2,75 @@ plugins {
id 'java'
id 'groovy'
id 'application'
id "com.github.johnrengelman.shadow" version "6.0.0"
id "net.nemerosa.versioning" version "2.14.0"
id "nebula.ospackage" version "8.4.1"
id 'jacoco'
id "net.nemerosa.versioning" version "2.15.1"
id "com.netflix.nebula.ospackage" version "11.5.0"
id "com.github.johnrengelman.shadow" version "7.1.2"
}
repositories {
jcenter()
mavenCentral()
}
dependencies {
compile 'info.picocli:picocli:4.5.1'
annotationProcessor 'info.picocli:picocli-codegen:4.5.1'
annotationProcessor 'info.picocli:picocli-codegen:4.7.5'
implementation 'info.picocli:picocli:4.7.5'
implementation 'org.slf4j:slf4j-api:2.0.9'
implementation 'org.slf4j:slf4j-simple:2.0.9'
implementation 'com.fasterxml.jackson.core:jackson-databind:2.15.2'
implementation 'com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.15.2'
implementation 'org.apache.commons:commons-collections4:4.4'
implementation 'org.slf4j:slf4j-api:1.7.+'
runtimeOnly 'ch.qos.logback:logback-classic:1.+'
testImplementation('org.spockframework:spock-core:2.0-M3-groovy-3.0')
testImplementation("org.slf4j:slf4j-simple:1.7.+")
testImplementation 'org.spockframework:spock-core:2.3-groovy-3.0'
}
application {
mainClassName = 'biz.nellemann.syslogd.SyslogServer'
getMainClass().set('biz.nellemann.syslogd.Application')
}
java {
sourceCompatibility = 1.8
targetCompatibility = 1.8
}
test {
useJUnitPlatform()
}
apply plugin: 'nebula.ospackage'
jacoco {
toolVersion = "0.8.10"
}
jacocoTestReport {
group = "verification"
reports {
xml.required = false
csv.required = false
html.destination file("${buildDir}/reports/coverage")
}
}
test.finalizedBy jacocoTestReport
jacocoTestCoverageVerification {
violationRules {
rule {
limit {
counter = 'LINE'
minimum = 0.3
}
limit {
counter = 'BRANCH'
minimum = 0.3
}
limit {
counter = 'CLASS'
minimum = 0.4
}
}
}
}
check.dependsOn jacocoTestCoverageVerification
ospackage {
packageName = 'syslogd'
release = '1'
@ -47,6 +87,10 @@ ospackage {
into 'bin'
}
from('doc/') {
into 'doc'
}
from(['README.md', 'LICENSE']) {
into 'doc'
}
@ -55,12 +99,11 @@ ospackage {
buildRpm {
dependsOn startShadowScripts
os = LINUX
os = "LINUX"
}
buildDeb {
dependsOn startShadowScripts
requires('default-jre-headless')
}
jar {
@ -76,3 +119,10 @@ jar {
)
}
}
tasks.register("packages") {
group "build"
dependsOn ":build"
dependsOn ":buildDeb"
dependsOn ":buildRpm"
}

88
doc/aix-errlogger.md Normal file
View File

@ -0,0 +1,88 @@
# AIX errlogger to remote syslog
Instructions for how to forward errlogger messages from IBM AIX and IBM Power Systems VIO Servers to a remote logging solution.
More information about the AIX errlogger is available on the IBM [knowledge center](https://www.ibm.com/support/knowledgecenter/ssw_aix_72/generalprogramming/error_notice.html).
## On each AIX / VIO Server
### Prepare the local syslog service
Configure the local syslog service to forward messages to our remote [syslogd](https://git.data.coop/nellemann/syslogd/) service.
Create an empty local log file:
```shell
touch /var/log/error.log
```
Add the following to the /etc/syslog.conf file:
```text
# Remote logging to remote host on port 514/UDP (AIX does not support non-default port number)
*.warn @10.32.64.1
# Also log to a local file, rotated daily and kept for 7 days
*.warn /var/log/error.log rotate time 1d files 7
# Optionally log authentication messages to remote host
#auth.info,authpriv.info @10.32.64.1
```
We use *10.32.64.1* as our remote syslog server in the above example.
Restart the syslogd service:
```shell
refresh -s syslogd
```
### Forward errlogger to the local syslog
We configure the AIX [error logger](https://www.ibm.com/docs/en/aix/7.3?topic=concepts-error-logging-overview) to forward messages to the local syslog service.
Create an odm errnotify logging template file:
```shell
cat << EOF >/tmp/err.tpl
errnotify:
en_name = "syslog1"
en_persistenceflg = 1
en_method = "/usr/bin/logger -plocal0.err [errnotify] seq: \$1 - \$(/usr/bin/errpt -l \$1 | tail -1)"
EOF
```
Add the template:
```shell
odmadd /tmp/err.tpl
```
Verify messages show up in the local syslog */var/log/error.log* file:
```shell
odmget -q"en_name='syslog1'" errnotify
errlogger system Test
```
#### Notes
If you need to delete the errnotify again:
```shell
odmdelete -o errnotify -q"en_name=syslog1"
```
To lookup err message details by a seq. no, run:
```shell
errpt -a -l [seq-no]
```
Or from the padmin shell:
```shell
errlog -ls -seq [seq-no]
```

24
doc/power-hmc.md Normal file
View File

@ -0,0 +1,24 @@
# Power Systems HMC Remote Logging
Instructions for how to forward syslog messages from a IBM Power Systems HMC to a remote logging solution.
More information about HMC logging is available on the IBM [knowledge center](https://www.ibm.com/support/pages/hmc-logging-and-auditing).
### Instructions
Network / Firewall must allow UDP (and possible TCP) traffic on port 514 from HMC to the remote syslog server. We use *10.32.64.1* as our remote syslog server in the example below.
To add a remote logging destination, run the following command. Use the filter to discard unwanted messaged.
```shell
chhmc -c syslog -t udp -s add -h 10.32.64.1 --input "filter_msg_contains_discard_strings=run-parts,slice,session,leases,renewal,0anacron,Session,DHCPREQUEST,DHCPACK,CMD,CRON,SCHEDULED"
```
In the above example we filter away some messages that we are not interested in forwarding on remotely.
To remove it again:
```shell
chhmc -c syslog -t udp -s remove -h 10.32.64.1
```

19
doc/powervc-auth.md Normal file
View File

@ -0,0 +1,19 @@
# PowerVC Remote Loggging
Configure rsyslog on your PowerVC instance to forward authentication messages to a remote logging solution. We use *10.32.64.1* as our remote syslog server in this example.
Create a file new file in the **/etc/rsyslog.d** folder (eg. *remote.conf*) with the following content:
```text
# Log all authentication messages to remote host
auth.* @10.32.64.1
# Log messages with severity warning (or above) to remote host
*.warn @10.32.64.1
```
Restart the rsyslog service
```shell
systemctl restart rsyslog
```

20
doc/readme-service.md Normal file
View File

@ -0,0 +1,20 @@
# Syslogd as a system service
## For systemd
To install as a systemd service, copy the [syslogd.service](syslogd.service)
file into */etc/systemd/system/*, edit the file and configure your required options.
Enable and start the service:
```shell
systemctl daemon-reload
systemctl enable syslogd.service
systemctl restart syslogd.service
```
To read log output from the service, use:
```shell
journalctl -f -u syslogd.service
```

16
doc/rsyslog-auth.md Normal file
View File

@ -0,0 +1,16 @@
# rsyslog
Configure rsyslog to forward authentication messages to a remote logging solution. We use *10.32.64.1* as our remote syslog server in this example.
Create a file new file in the **/etc/rsyslog.d** folder (eg. *90-auth.conf*) with the following content:
```text
# Log authentication messages to remote host
auth.*,authpriv.* @10.32.64.1
```
Restart the rsyslog service
```shell
systemctl restart rsyslog
```

1
doc/syslogd.drawio Normal file
View File

@ -0,0 +1 @@
<mxfile host="drawio-plugin" modified="2023-02-05T13:04:08.556Z" agent="5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.102 Safari/537.36" etag="__kq9uG-1g-sjP8t85Xj" version="20.5.3" type="embed"><diagram id="23iRSUPoRavnBvh4doch" name="Page-1"><mxGraphModel dx="809" dy="749" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="1169" pageHeight="827" math="0" shadow="0"><root><mxCell id="0"/><mxCell id="1" parent="0"/><mxCell id="10" style="edgeStyle=orthogonalEdgeStyle;orthogonalLoop=1;jettySize=auto;html=1;curved=1;sketch=1;shadow=1;" parent="1" source="2" target="3" edge="1"><mxGeometry relative="1" as="geometry"/></mxCell><mxCell id="2" value="Syslog&lt;br&gt;RFC--3164" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#ffe6cc;strokeColor=#d79b00;" parent="1" vertex="1"><mxGeometry x="50" y="90" width="120" height="60" as="geometry"/></mxCell><mxCell id="13" style="edgeStyle=orthogonalEdgeStyle;curved=1;sketch=1;orthogonalLoop=1;jettySize=auto;html=1;shadow=1;" parent="1" source="3" target="9" edge="1"><mxGeometry relative="1" as="geometry"/></mxCell><mxCell id="14" style="edgeStyle=orthogonalEdgeStyle;curved=1;sketch=1;orthogonalLoop=1;jettySize=auto;html=1;shadow=1;" parent="1" source="3" target="6" edge="1"><mxGeometry relative="1" as="geometry"/></mxCell><mxCell id="15" style="edgeStyle=orthogonalEdgeStyle;curved=1;sketch=1;orthogonalLoop=1;jettySize=auto;html=1;shadow=1;" parent="1" source="3" target="7" edge="1"><mxGeometry relative="1" as="geometry"/></mxCell><mxCell id="16" style="edgeStyle=orthogonalEdgeStyle;curved=1;sketch=1;orthogonalLoop=1;jettySize=auto;html=1;shadow=1;" parent="1" source="3" target="8" edge="1"><mxGeometry relative="1" as="geometry"/></mxCell><mxCell id="3" value="syslogd" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;sketch=1;rounded=1;shadow=1;fillColor=#dae8fc;strokeColor=#6c8ebf;" parent="1" vertex="1"><mxGeometry x="280" y="130" width="120" height="60" as="geometry"/></mxCell><mxCell id="11" style="edgeStyle=orthogonalEdgeStyle;curved=1;sketch=1;orthogonalLoop=1;jettySize=auto;html=1;shadow=1;" parent="1" source="4" target="3" edge="1"><mxGeometry relative="1" as="geometry"/></mxCell><mxCell id="4" value="Syslog&lt;br&gt;RFC--5424" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#fff2cc;strokeColor=#d6b656;" parent="1" vertex="1"><mxGeometry x="40" y="180" width="120" height="60" as="geometry"/></mxCell><mxCell id="12" style="edgeStyle=orthogonalEdgeStyle;curved=1;sketch=1;orthogonalLoop=1;jettySize=auto;html=1;shadow=1;" parent="1" source="5" target="3" edge="1"><mxGeometry relative="1" as="geometry"/></mxCell><mxCell id="5" value="GELF" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#f8cecc;strokeColor=#b85450;" parent="1" vertex="1"><mxGeometry x="120" y="260" width="120" height="60" as="geometry"/></mxCell><mxCell id="6" value="Syslog&lt;br&gt;RFC-5424" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#d5e8d4;strokeColor=#82b366;" parent="1" vertex="1"><mxGeometry x="570" y="80" width="120" height="60" as="geometry"/></mxCell><mxCell id="7" value="Grafana Loki" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#e1d5e7;strokeColor=#9673a6;" parent="1" vertex="1"><mxGeometry x="550" y="170" width="120" height="60" as="geometry"/></mxCell><mxCell id="8" value="Graylog" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#fff2cc;strokeColor=#d6b656;" parent="1" vertex="1"><mxGeometry x="420" y="250" width="120" height="60" as="geometry"/></mxCell><mxCell id="9" value="Standard&lt;br&gt;Output" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#f5f5f5;strokeColor=#666666;fontColor=#333333;" parent="1" vertex="1"><mxGeometry x="410" y="30" width="120" height="60" as="geometry"/></mxCell><mxCell id="17" value="Standard&lt;br&gt;Input" style="rounded=1;whiteSpace=wrap;html=1;sketch=1;shadow=1;fillColor=#f5f5f5;strokeColor=#666666;fontColor=#333333;" vertex="1" parent="1"><mxGeometry x="180" y="30" width="90" height="60" as="geometry"/></mxCell><mxCell id="19" style="edgeStyle=orthogonalEdgeStyle;orthogonalLoop=1;jettySize=auto;html=1;curved=1;sketch=1;shadow=1;" edge="1" parent="1" source="17" target="3"><mxGeometry relative="1" as="geometry"><mxPoint x="190" y="110" as="sourcePoint"/><mxPoint x="300" y="170" as="targetPoint"/></mxGeometry></mxCell></root></mxGraphModel></diagram></mxfile>

BIN
doc/syslogd.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 96 KiB

10
doc/syslogd.service Normal file
View File

@ -0,0 +1,10 @@
[Unit]
Description=Syslog Director
[Service]
TimeoutStartSec=0
Restart=always
ExecStart=/opt/syslogd/bin/syslogd --port 514 --no-ansi
[Install]
WantedBy=default.target

View File

@ -1,3 +1,5 @@
id = syslogd
name = syslogd
group = biz.nellemann.syslogd
version = 1.0.4
version = 1.3.5
description = "Syslog Director"

Binary file not shown.

View File

@ -1,5 +1,6 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-6.6.1-bin.zip
distributionUrl=https\://services.gradle.org/distributions/gradle-7.6.2-bin.zip
networkTimeout=10000
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists

285
gradlew vendored
View File

@ -1,7 +1,7 @@
#!/usr/bin/env sh
#!/bin/sh
#
# Copyright 2015 the original author or authors.
# Copyright © 2015-2021 the original authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@ -17,67 +17,101 @@
#
##############################################################################
##
## Gradle start up script for UN*X
##
#
# Gradle start up script for POSIX generated by Gradle.
#
# Important for running:
#
# (1) You need a POSIX-compliant shell to run this script. If your /bin/sh is
# noncompliant, but you have some other compliant shell such as ksh or
# bash, then to run this script, type that shell name before the whole
# command line, like:
#
# ksh Gradle
#
# Busybox and similar reduced shells will NOT work, because this script
# requires all of these POSIX shell features:
# * functions;
# * expansions «$var», «${var}», «${var:-default}», «${var+SET}»,
# «${var#prefix}», «${var%suffix}», and «$( cmd )»;
# * compound commands having a testable exit status, especially «case»;
# * various built-in commands including «command», «set», and «ulimit».
#
# Important for patching:
#
# (2) This script targets any POSIX shell, so it avoids extensions provided
# by Bash, Ksh, etc; in particular arrays are avoided.
#
# The "traditional" practice of packing multiple parameters into a
# space-separated string is a well documented source of bugs and security
# problems, so this is (mostly) avoided, by progressively accumulating
# options in "$@", and eventually passing that to Java.
#
# Where the inherited environment variables (DEFAULT_JVM_OPTS, JAVA_OPTS,
# and GRADLE_OPTS) rely on word-splitting, this is performed explicitly;
# see the in-line comments for details.
#
# There are tweaks for specific operating systems such as AIX, CygWin,
# Darwin, MinGW, and NonStop.
#
# (3) This script is generated from the Groovy template
# https://github.com/gradle/gradle/blob/HEAD/subprojects/plugins/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt
# within the Gradle project.
#
# You can find Gradle at https://github.com/gradle/gradle/.
#
##############################################################################
# Attempt to set APP_HOME
# Resolve links: $0 may be a link
PRG="$0"
# Need this for relative symlinks.
while [ -h "$PRG" ] ; do
ls=`ls -ld "$PRG"`
link=`expr "$ls" : '.*-> \(.*\)$'`
if expr "$link" : '/.*' > /dev/null; then
PRG="$link"
else
PRG=`dirname "$PRG"`"/$link"
fi
done
SAVED="`pwd`"
cd "`dirname \"$PRG\"`/" >/dev/null
APP_HOME="`pwd -P`"
cd "$SAVED" >/dev/null
APP_NAME="Gradle"
APP_BASE_NAME=`basename "$0"`
# Resolve links: $0 may be a link
app_path=$0
# Need this for daisy-chained symlinks.
while
APP_HOME=${app_path%"${app_path##*/}"} # leaves a trailing /; empty if no leading path
[ -h "$app_path" ]
do
ls=$( ls -ld "$app_path" )
link=${ls#*' -> '}
case $link in #(
/*) app_path=$link ;; #(
*) app_path=$APP_HOME$link ;;
esac
done
# This is normally unused
# shellcheck disable=SC2034
APP_BASE_NAME=${0##*/}
APP_HOME=$( cd "${APP_HOME:-./}" && pwd -P ) || exit
# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
# Use the maximum available, or set MAX_FD != -1 to use that value.
MAX_FD="maximum"
MAX_FD=maximum
warn () {
echo "$*"
}
} >&2
die () {
echo
echo "$*"
echo
exit 1
}
} >&2
# OS specific support (must be 'true' or 'false').
cygwin=false
msys=false
darwin=false
nonstop=false
case "`uname`" in
CYGWIN* )
cygwin=true
;;
Darwin* )
darwin=true
;;
MINGW* )
msys=true
;;
NONSTOP* )
nonstop=true
;;
case "$( uname )" in #(
CYGWIN* ) cygwin=true ;; #(
Darwin* ) darwin=true ;; #(
MSYS* | MINGW* ) msys=true ;; #(
NONSTOP* ) nonstop=true ;;
esac
CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar
@ -87,9 +121,9 @@ CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar
if [ -n "$JAVA_HOME" ] ; then
if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
# IBM's JDK on AIX uses strange locations for the executables
JAVACMD="$JAVA_HOME/jre/sh/java"
JAVACMD=$JAVA_HOME/jre/sh/java
else
JAVACMD="$JAVA_HOME/bin/java"
JAVACMD=$JAVA_HOME/bin/java
fi
if [ ! -x "$JAVACMD" ] ; then
die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME
@ -98,7 +132,7 @@ Please set the JAVA_HOME variable in your environment to match the
location of your Java installation."
fi
else
JAVACMD="java"
JAVACMD=java
which java >/dev/null 2>&1 || die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
Please set the JAVA_HOME variable in your environment to match the
@ -106,80 +140,105 @@ location of your Java installation."
fi
# Increase the maximum file descriptors if we can.
if [ "$cygwin" = "false" -a "$darwin" = "false" -a "$nonstop" = "false" ] ; then
MAX_FD_LIMIT=`ulimit -H -n`
if [ $? -eq 0 ] ; then
if [ "$MAX_FD" = "maximum" -o "$MAX_FD" = "max" ] ; then
MAX_FD="$MAX_FD_LIMIT"
fi
ulimit -n $MAX_FD
if [ $? -ne 0 ] ; then
warn "Could not set maximum file descriptor limit: $MAX_FD"
fi
else
warn "Could not query maximum file descriptor limit: $MAX_FD_LIMIT"
fi
fi
# For Darwin, add options to specify how the application appears in the dock
if $darwin; then
GRADLE_OPTS="$GRADLE_OPTS \"-Xdock:name=$APP_NAME\" \"-Xdock:icon=$APP_HOME/media/gradle.icns\""
fi
# For Cygwin or MSYS, switch paths to Windows format before running java
if [ "$cygwin" = "true" -o "$msys" = "true" ] ; then
APP_HOME=`cygpath --path --mixed "$APP_HOME"`
CLASSPATH=`cygpath --path --mixed "$CLASSPATH"`
JAVACMD=`cygpath --unix "$JAVACMD"`
# We build the pattern for arguments to be converted via cygpath
ROOTDIRSRAW=`find -L / -maxdepth 1 -mindepth 1 -type d 2>/dev/null`
SEP=""
for dir in $ROOTDIRSRAW ; do
ROOTDIRS="$ROOTDIRS$SEP$dir"
SEP="|"
done
OURCYGPATTERN="(^($ROOTDIRS))"
# Add a user-defined pattern to the cygpath arguments
if [ "$GRADLE_CYGPATTERN" != "" ] ; then
OURCYGPATTERN="$OURCYGPATTERN|($GRADLE_CYGPATTERN)"
fi
# Now convert the arguments - kludge to limit ourselves to /bin/sh
i=0
for arg in "$@" ; do
CHECK=`echo "$arg"|egrep -c "$OURCYGPATTERN" -`
CHECK2=`echo "$arg"|egrep -c "^-"` ### Determine if an option
if [ $CHECK -ne 0 ] && [ $CHECK2 -eq 0 ] ; then ### Added a condition
eval `echo args$i`=`cygpath --path --ignore --mixed "$arg"`
else
eval `echo args$i`="\"$arg\""
fi
i=`expr $i + 1`
done
case $i in
0) set -- ;;
1) set -- "$args0" ;;
2) set -- "$args0" "$args1" ;;
3) set -- "$args0" "$args1" "$args2" ;;
4) set -- "$args0" "$args1" "$args2" "$args3" ;;
5) set -- "$args0" "$args1" "$args2" "$args3" "$args4" ;;
6) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" ;;
7) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" ;;
8) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" ;;
9) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" "$args8" ;;
if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then
case $MAX_FD in #(
max*)
# In POSIX sh, ulimit -H is undefined. That's why the result is checked to see if it worked.
# shellcheck disable=SC3045
MAX_FD=$( ulimit -H -n ) ||
warn "Could not query maximum file descriptor limit"
esac
case $MAX_FD in #(
'' | soft) :;; #(
*)
# In POSIX sh, ulimit -n is undefined. That's why the result is checked to see if it worked.
# shellcheck disable=SC3045
ulimit -n "$MAX_FD" ||
warn "Could not set maximum file descriptor limit to $MAX_FD"
esac
fi
# Escape application args
save () {
for i do printf %s\\n "$i" | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/' \\\\/" ; done
echo " "
}
APP_ARGS=`save "$@"`
# Collect all arguments for the java command, stacking in reverse order:
# * args from the command line
# * the main class name
# * -classpath
# * -D...appname settings
# * --module-path (only if needed)
# * DEFAULT_JVM_OPTS, JAVA_OPTS, and GRADLE_OPTS environment variables.
# Collect all arguments for the java command, following the shell quoting and substitution rules
eval set -- $DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS "\"-Dorg.gradle.appname=$APP_BASE_NAME\"" -classpath "\"$CLASSPATH\"" org.gradle.wrapper.GradleWrapperMain "$APP_ARGS"
# For Cygwin or MSYS, switch paths to Windows format before running java
if "$cygwin" || "$msys" ; then
APP_HOME=$( cygpath --path --mixed "$APP_HOME" )
CLASSPATH=$( cygpath --path --mixed "$CLASSPATH" )
JAVACMD=$( cygpath --unix "$JAVACMD" )
# Now convert the arguments - kludge to limit ourselves to /bin/sh
for arg do
if
case $arg in #(
-*) false ;; # don't mess with options #(
/?*) t=${arg#/} t=/${t%%/*} # looks like a POSIX filepath
[ -e "$t" ] ;; #(
*) false ;;
esac
then
arg=$( cygpath --path --ignore --mixed "$arg" )
fi
# Roll the args list around exactly as many times as the number of
# args, so each arg winds up back in the position where it started, but
# possibly modified.
#
# NB: a `for` loop captures its iteration list before it begins, so
# changing the positional parameters here affects neither the number of
# iterations, nor the values presented in `arg`.
shift # remove old arg
set -- "$@" "$arg" # push replacement arg
done
fi
# Collect all arguments for the java command;
# * $DEFAULT_JVM_OPTS, $JAVA_OPTS, and $GRADLE_OPTS can contain fragments of
# shell script including quotes and variable substitutions, so put them in
# double quotes to make sure that they get re-expanded; and
# * put everything else in single quotes, so that it's not re-expanded.
set -- \
"-Dorg.gradle.appname=$APP_BASE_NAME" \
-classpath "$CLASSPATH" \
org.gradle.wrapper.GradleWrapperMain \
"$@"
# Stop when "xargs" is not available.
if ! command -v xargs >/dev/null 2>&1
then
die "xargs is not available"
fi
# Use "xargs" to parse quoted args.
#
# With -n1 it outputs one arg per line, with the quotes and backslashes removed.
#
# In Bash we could simply go:
#
# readarray ARGS < <( xargs -n1 <<<"$var" ) &&
# set -- "${ARGS[@]}" "$@"
#
# but POSIX shell has neither arrays nor command substitution, so instead we
# post-process each arg (as a line of input to sed) to backslash-escape any
# character that might be a shell metacharacter, then use eval to reverse
# that process (while maintaining the separation between arguments), and wrap
# the whole thing up as a single "set" statement.
#
# This will of course break if any of these variables contains a newline or
# an unmatched quote.
#
eval "set -- $(
printf '%s\n' "$DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS" |
xargs -n1 |
sed ' s~[^-[:alnum:]+,./:=@_]~\\&~g; ' |
tr '\n' ' '
)" '"$@"'
exec "$JAVACMD" "$@"

15
gradlew.bat vendored
View File

@ -14,7 +14,7 @@
@rem limitations under the License.
@rem
@if "%DEBUG%" == "" @echo off
@if "%DEBUG%"=="" @echo off
@rem ##########################################################################
@rem
@rem Gradle startup script for Windows
@ -25,7 +25,8 @@
if "%OS%"=="Windows_NT" setlocal
set DIRNAME=%~dp0
if "%DIRNAME%" == "" set DIRNAME=.
if "%DIRNAME%"=="" set DIRNAME=.
@rem This is normally unused
set APP_BASE_NAME=%~n0
set APP_HOME=%DIRNAME%
@ -40,7 +41,7 @@ if defined JAVA_HOME goto findJavaFromJavaHome
set JAVA_EXE=java.exe
%JAVA_EXE% -version >NUL 2>&1
if "%ERRORLEVEL%" == "0" goto execute
if %ERRORLEVEL% equ 0 goto execute
echo.
echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
@ -75,13 +76,15 @@ set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar
:end
@rem End local scope for the variables with windows NT shell
if "%ERRORLEVEL%"=="0" goto mainEnd
if %ERRORLEVEL% equ 0 goto mainEnd
:fail
rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of
rem the _cmd.exe /c_ return code!
if not "" == "%GRADLE_EXIT_CONSOLE%" exit 1
exit /b 1
set EXIT_CODE=%ERRORLEVEL%
if %EXIT_CODE% equ 0 set EXIT_CODE=1
if not ""=="%GRADLE_EXIT_CONSOLE%" exit %EXIT_CODE%
exit /b %EXIT_CODE%
:mainEnd
if "%OS%"=="Windows_NT" endlocal

View File

@ -1,10 +1 @@
/*
* This file was generated by the Gradle 'init' task.
*
* The settings file is used to specify which projects to include in your build.
*
* Detailed information about configuring a multi-project build in Gradle can be found
* in the user manual at https://docs.gradle.org/6.6.1/userguide/multi_project_builds.html
*/
rootProject.name = 'syslogd'

View File

@ -1,3 +1,18 @@
/*
Copyright 2021 mark.nellemann@gmail.com
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package biz.nellemann.syslogd;
public enum Ansi {

View File

@ -0,0 +1,193 @@
/*
Copyright 2020 mark.nellemann@gmail.com
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package biz.nellemann.syslogd;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.net.URI;
import java.net.URL;
import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
import java.util.concurrent.Callable;
import biz.nellemann.syslogd.msg.SyslogMessage;
import biz.nellemann.syslogd.net.GelfClient;
import biz.nellemann.syslogd.net.LokiClient;
import biz.nellemann.syslogd.net.TcpServer;
import biz.nellemann.syslogd.net.UdpClient;
import biz.nellemann.syslogd.net.UdpServer;
import biz.nellemann.syslogd.parser.GelfParser;
import biz.nellemann.syslogd.parser.SyslogParser;
import biz.nellemann.syslogd.parser.SyslogParserRfc3164;
import biz.nellemann.syslogd.parser.SyslogParserRfc5424;
import picocli.CommandLine;
import picocli.CommandLine.Command;
@Command(name = "syslogd",
mixinStandardHelpOptions = true,
versionProvider = biz.nellemann.syslogd.VersionProvider.class)
public class Application implements Callable<Integer>, LogReceiveListener {
private final List<LogForwardListener> logForwardListeners = new ArrayList<>();
private SyslogParser syslogParser;
@CommandLine.Option(names = {"-p", "--port"}, description = "Listening port [default: 1514].", defaultValue = "1514", paramLabel = "<num>")
private int port;
@CommandLine.Option(names = "--no-udp", negatable = true, description = "Listen on UDP [default: true].", defaultValue = "true")
private boolean udpServer;
@CommandLine.Option(names = "--no-tcp", negatable = true, description = "Listen on TCP [default: true].", defaultValue = "true")
private boolean tcpServer;
@CommandLine.Option(names = "--no-ansi", negatable = true, description = "Output in ANSI colors [default: true].", defaultValue = "true")
private boolean ansiOutput;
@CommandLine.Option(names = "--no-stdout", negatable = true, description = "Output messages to stdout [default: true].", defaultValue = "true")
private boolean stdout;
@CommandLine.Option(names = "--no-stdin", negatable = true, description = "Forward messages from stdin [default: true].", defaultValue = "true")
private boolean stdin;
@CommandLine.Option(names = {"-f", "--format"}, description = "Input format: RFC-5424, RFC-3164 or GELF [default: RFC-3164].", defaultValue = "RFC-3164")
private String protocol;
@CommandLine.Option(names = { "--to-syslog"}, description = "Forward to Syslog <udp://host:port> (RFC-5424).", paramLabel = "<uri>")
private URI syslog;
@CommandLine.Option(names = { "--to-gelf"}, description = "Forward to Graylog <udp://host:port>.", paramLabel = "<uri>")
private URI gelf;
@CommandLine.Option(names = { "--to-loki"}, description = "Forward to Grafana Loki <http://host:port>.", paramLabel = "<url>")
private URL loki;
@CommandLine.Option(names = { "-d", "--debug" }, description = "Enable debugging [default: 'false'].")
private boolean enableDebug = false;
@Override
public Integer call() throws IOException {
if(enableDebug) {
System.setProperty("org.slf4j.simpleLogger.defaultLogLevel", "DEBUG");
}
if(protocol.equalsIgnoreCase("GELF"))
syslogParser = new GelfParser();
else if (protocol.equalsIgnoreCase("RFC-5424")) {
syslogParser = new SyslogParserRfc5424();
} else {
syslogParser = new SyslogParserRfc3164();
}
if(syslog != null) {
if(syslog.getScheme().toLowerCase(Locale.ROOT).equals("udp")) {
UdpClient udpClient = new UdpClient(getInetSocketAddress(syslog));
logForwardListeners.add(udpClient);
} else {
throw new UnsupportedOperationException("Forward protocol not implemented: " + syslog.getScheme());
}
}
if(gelf != null) {
if(gelf.getScheme().toLowerCase(Locale.ROOT).equals("udp")) {
GelfClient gelfClient = new GelfClient(getInetSocketAddress(gelf));
logForwardListeners.add(gelfClient);
} else {
throw new UnsupportedOperationException("Forward protocol not implemented: " + gelf.getScheme());
}
}
if(loki != null) {
LokiClient lokiClient = new LokiClient(loki);
logForwardListeners.add(lokiClient);
Thread t = new Thread(lokiClient);
t.start();
}
if(stdin) {
InputReader inputReader = new InputReader(System.in, protocol);
inputReader.addEventListener(this);
inputReader.start();
}
if(udpServer) {
UdpServer udpServer = new UdpServer(port);
udpServer.addEventListener(this);
udpServer.start();
}
if(tcpServer) {
TcpServer tcpServer = new TcpServer(port);
tcpServer.addEventListener(this);
tcpServer.start();
}
return 0;
}
@Override
public void onLogEvent(LogReceiveEvent event) {
// Parse message
SyslogMessage msg = null;
try {
msg = syslogParser.parse(event.getBytes());
} catch(Exception e) {
e.printStackTrace();
}
if(msg != null) {
if(!logForwardListeners.isEmpty()) {
sendForwardEvent(msg);
}
if(stdout) {
if(ansiOutput) {
System.out.println(SyslogPrinter.toAnsiString(msg));
} else {
System.out.println(SyslogPrinter.toString(msg));
}
}
}
}
private void sendForwardEvent(SyslogMessage message) {
LogForwardEvent event = new LogForwardEvent( this, message);
for (LogForwardListener listener : logForwardListeners) {
listener.onForwardEvent(event);
}
}
private InetSocketAddress getInetSocketAddress(URI input) {
return new InetSocketAddress(input.getHost(), input.getPort());
}
public static void main(String... args) {
int exitCode = new CommandLine(new Application()).execute(args);
System.exit(exitCode);
}
}

View File

@ -0,0 +1,70 @@
package biz.nellemann.syslogd;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.List;
import java.util.Scanner;
import biz.nellemann.syslogd.msg.SyslogMessage;
public class InputReader extends Thread {
private final Scanner input;
private final String protocol;
public InputReader(InputStream inputStream, String protocol) {
input = new Scanner(inputStream);
this.protocol = protocol;
}
@Override
public void run() {
while(input.hasNextLine()) {
SyslogMessage msg = new SyslogMessage(input.nextLine());
msg.hostname = "localhost";
msg.application = "syslogd";
String payload;
if(protocol.equalsIgnoreCase("GELF"))
payload = SyslogPrinter.toGelf(msg);
else if (protocol.equalsIgnoreCase("RFC-5424")) {
payload = SyslogPrinter.toRfc5424(msg);
} else {
payload = SyslogPrinter.toRfc3164(msg);
}
sendEvent(payload);
}
input.close();
}
private synchronized void sendEvent(String text) {
LogReceiveEvent event = new LogReceiveEvent( this, text);
for (LogReceiveListener eventListener : eventListeners) {
eventListener.onLogEvent(event);
}
}
/**
* Event Listener Configuration
*/
protected List<LogReceiveListener> eventListeners = new ArrayList<>();
public synchronized void addEventListener(LogReceiveListener listener ) {
eventListeners.add( listener );
}
public synchronized void addEventListener(List<LogReceiveListener> listeners ) {
eventListeners.addAll(listeners);
}
public synchronized void removeEventListener( LogReceiveListener l ) {
eventListeners.remove( l );
}
}

View File

@ -1,5 +1,5 @@
/*
Copyright 2020 mark.nellemann@gmail.com
Copyright 2021 mark.nellemann@gmail.com
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
@ -15,19 +15,21 @@
*/
package biz.nellemann.syslogd;
import biz.nellemann.syslogd.msg.SyslogMessage;
import java.util.EventObject;
public class LogEvent extends EventObject {
public class LogForwardEvent extends EventObject {
private final String message;
private static final long serialVersionUID = 1L;
private final SyslogMessage message;
public LogEvent(final Object source, final String message ) {
public LogForwardEvent(final Object source, final SyslogMessage message ) {
super( source );
this.message = message;
}
public String getMessage() {
public SyslogMessage getMessage() {
return message;
}
}

View File

@ -0,0 +1,20 @@
/*
Copyright 2021 mark.nellemann@gmail.com
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package biz.nellemann.syslogd;
public interface LogForwardListener {
public void onForwardEvent(LogForwardEvent event);
}

View File

@ -0,0 +1,47 @@
/*
Copyright 2020 mark.nellemann@gmail.com
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package biz.nellemann.syslogd;
import java.net.DatagramPacket;
import java.nio.charset.StandardCharsets;
import java.util.Arrays;
import java.util.EventObject;
public class LogReceiveEvent extends EventObject {
private static final long serialVersionUID = 1L;
private final DatagramPacket packet;
public LogReceiveEvent(final Object source, final String message ) {
super( source );
byte[] bytes = message.getBytes();
this.packet = new DatagramPacket(bytes, bytes.length);
}
public LogReceiveEvent(final Object source, final DatagramPacket packet) {
super( source );
this.packet = packet;
}
public String getText() {
return new String(packet.getData(), packet.getOffset(), packet.getLength(), StandardCharsets.UTF_8);
}
public byte[] getBytes() {
return Arrays.copyOfRange(packet.getData(), packet.getOffset(), packet.getLength());
}
}

View File

@ -15,6 +15,6 @@
*/
package biz.nellemann.syslogd;
public interface LogListener {
void onLogEvent(LogEvent event);
public interface LogReceiveListener {
public void onLogEvent(LogReceiveEvent event);
}

View File

@ -1,200 +0,0 @@
/*
Copyright 2020 mark.nellemann@gmail.com
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package biz.nellemann.syslogd;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.time.*;
import java.time.format.DateTimeFormatter;
import java.time.format.DateTimeParseException;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
public class SyslogParser {
private final static Logger log = LoggerFactory.getLogger(SyslogParser.class);
/**
* Parses [rfc3164](https://tools.ietf.org/html/rfc3164) syslog messages.
*
* @param input
* @return
* @throws NumberFormatException
*/
public static SyslogMessage parseRfc3164(final String input) throws NumberFormatException {
Pattern pattern = Pattern.compile("^<(\\d{1,3})>(\\D{3}\\s+\\d{1,2} \\d{2}:\\d{2}:\\d{2})\\s+(Message forwarded from \\S+:|\\S+)\\s+([^\\s:]+):?\\s+(.*)", Pattern.CASE_INSENSITIVE);
Matcher matcher = pattern.matcher(input);
boolean matchFound = matcher.find();
if(!matchFound) {
//log.warn("parseRfc3164() - Match not found in: ");
System.err.println("!" + input);
return null;
}
String pri = matcher.group(1);
String date = matcher.group(2);
String hostname = matcher.group(3);
String application = matcher.group(4);
String msg = matcher.group(5);
if(hostname.endsWith(":")) {
String[] tmp = hostname.split(" ");
hostname = tmp[tmp.length-1];
hostname = hostname.substring(0, hostname.length()-1);
}
Integer facility = getFacility(pri);
Integer severity = getSeverity(pri);
SyslogMessage syslogMessage = new SyslogMessage(msg.trim());
syslogMessage.facility = Facility.getByNumber(facility);
syslogMessage.severity = Severity.getByNumber(severity);
syslogMessage.timestamp = parseRfc3164Timestamp(date);
syslogMessage.hostname = hostname;
syslogMessage.application = application;
return syslogMessage;
}
/**
* Parses [rfc5424](https://tools.ietf.org/html/rfc5424) syslog messages.
*
* @param input
* @return
* @throws NumberFormatException
*/
public static SyslogMessage parseRfc5424(final String input) throws NumberFormatException {
Pattern pattern = Pattern.compile("^<(\\d{1,3})>(\\d+)\\s+(\\S+)\\s+(\\S+)\\s+(\\S+)\\s+(\\S+)\\s+(\\S+)\\s+(\\[.*\\])\\s+(\\S+)", Pattern.CASE_INSENSITIVE);
Matcher matcher = pattern.matcher(input);
boolean matchFound = matcher.find();
if(!matchFound) {
//log.warn("parseRfc5424() - Match not found in: " + input);
System.err.println("!" + input);
return null;
}
String pri = matcher.group(1);
String ver = matcher.group(2);
String date = matcher.group(3);
String host = matcher.group(4);
String app = matcher.group(5);
String procId = matcher.group(6);
String msgId = matcher.group(7);
String data = matcher.group(8);
String msg = matcher.group(9);
Integer facility = getFacility(pri);
Integer severity = getSeverity(pri);
SyslogMessage syslogMessage = new SyslogMessage(msg.trim());
syslogMessage.facility = Facility.getByNumber(facility);
syslogMessage.severity = Severity.getByNumber(severity);
syslogMessage.version = Integer.parseInt(ver);
syslogMessage.timestamp = parseRfc5424Timestamp(date);
syslogMessage.hostname = host;
if(app != null && !app.equals("-"))
syslogMessage.application = app;
if(procId != null && !procId.equals("-"))
syslogMessage.processId = procId;
if(msgId != null && !msgId.equals("-"))
syslogMessage.messageId = msgId;
syslogMessage.structuredData = data;
return syslogMessage;
}
/**
* Parse rfc3164 TIMESTAMP field into Instant.
*
* @param dateString
* @return
*/
static protected Instant parseRfc3164Timestamp(String dateString) {
// We need to add year to parse date correctly
OffsetDateTime odt = OffsetDateTime.now();
// Date: Mmm dd hh:mm:ss
Instant instant = null;
try {
DateTimeFormatter dateTimeFormatter = DateTimeFormatter.ofPattern("yyyy MMM [ ]d HH:mm:ss").withZone(ZoneOffset.UTC);
instant = Instant.from(dateTimeFormatter.parse(odt.getYear() + " " + dateString));
} catch(DateTimeParseException e) {
log.error("parseDate()", e);
}
return instant;
}
/**
* Parse rfc5424 TIMESTAMP field into Instant.
*
* @param dateString
* @return
*/
static protected Instant parseRfc5424Timestamp(String dateString) {
Instant instant = null;
try {
DateTimeFormatter dateTimeFormatter = DateTimeFormatter.ISO_OFFSET_DATE_TIME;
instant = Instant.from(dateTimeFormatter.parse(dateString));
} catch(DateTimeParseException e) {
log.error("parseTimestamp()", e);
}
return instant;
}
/**
* Converts syslog PRI field into Facility.
*
* @param pri
* @return
*/
static protected int getFacility(String pri) {
int priority = Integer.parseInt(pri);
int facility = priority >> 3;
log.debug("getFacility() - " + pri + " => " + facility);
return facility;
}
/**
* Converts syslog PRI field into Severity.
*
* @param pri
* @return
*/
static protected int getSeverity(String pri) {
int priority = Integer.parseInt(pri);
int severity = priority & 0x07;
log.debug("getSeverity() - " + pri + " => " + severity);
return severity;
}
}

View File

@ -0,0 +1,158 @@
/*
Copyright 2021 mark.nellemann@gmail.com
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package biz.nellemann.syslogd;
import biz.nellemann.syslogd.msg.Facility;
import biz.nellemann.syslogd.msg.Severity;
import biz.nellemann.syslogd.msg.SyslogMessage;
import biz.nellemann.syslogd.parser.JsonUtil;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class SyslogPrinter {
private final static Logger log = LoggerFactory.getLogger(SyslogPrinter.class);
private final static char SPACE = ' ';
public static String toString(SyslogMessage msg) {
StringBuilder sb = new StringBuilder(msg.timestamp.toString());
sb.append(String.format(" [%8.8s.%-6.6s] ", msg.facility, msg.severity));
sb.append(String.format(" %-16.16s ", msg.hostname));
sb.append(String.format(" %-16.16s ", msg.application));
sb.append(msg.message);
return sb.toString();
}
public static String toAnsiString(SyslogMessage msg) {
StringBuilder sb = new StringBuilder(msg.timestamp.toString());
if (msg.severity.toNumber() < 3) {
sb.append(Ansi.RED);
} else if (msg.severity.toNumber() < 5) {
sb.append(Ansi.YELLOW);
} else {
sb.append(Ansi.GREEN);
}
sb.append(String.format(" [%8.8s.%-6.6s] ", msg.facility, msg.severity)).append(Ansi.RESET);
sb.append(Ansi.BLUE).append(String.format(" %-16.16s ", msg.hostname)).append(Ansi.RESET);
sb.append(Ansi.CYAN).append(String.format(" %-16.16s ", msg.application)).append(Ansi.RESET);
sb.append(msg.message);
return sb.toString();
}
/**
* Return a RFC-3164 formatted string of the SyslogMessage.
* @param msg
* @return
*/
public static String toRfc3164(SyslogMessage msg) {
StringBuilder sb = new StringBuilder();
sb.append(getPri(msg.facility, msg.severity));
sb.append(new java.text.SimpleDateFormat("MMM dd HH:mm:ss").format(new java.util.Date(msg.timestamp.toEpochMilli())));
sb.append(SPACE).append(msg.hostname);
sb.append(SPACE).append(msg.application);
sb.append(":").append(SPACE).append(msg.message);
log.debug(sb.toString());
return sb.toString();
}
/**
* Return a RFC-5424 formatted string of the SyslogMessage.
* @param msg
* @return
*/
public static String toRfc5424(SyslogMessage msg) {
StringBuilder sb = new StringBuilder();
sb.append(getPri(msg.facility, msg.severity)).append("1");
sb.append(SPACE).append(new java.text.SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'").format(new java.util.Date(msg.timestamp.toEpochMilli())));
sb.append(SPACE).append(msg.hostname);
sb.append(SPACE).append(msg.application);
sb.append(SPACE).append(msg.processId != null ? msg.processId : "-");
sb.append(SPACE).append(msg.messageId != null ? msg.messageId : "-");
sb.append(SPACE).append(msg.structuredData != null ? msg.structuredData : "-");
sb.append(SPACE).append(msg.message);
log.debug(sb.toString());
return sb.toString();
}
/**
* Return a GELF JSON formatted string of the SyslogMessage.
* https://www.graylog.org/features/gelf
* @param msg
* @return
*/
public static String toGelf(SyslogMessage msg) {
StringBuilder sb = new StringBuilder("{ \"version\": \"1.1\"");
sb.append(String.format(", \"host\": \"%s\"", msg.hostname));
sb.append(String.format(", \"short_message\": \"%s\"", JsonUtil.encode(msg.message)));
sb.append(String.format(", \"full_message\": \"%s\"", JsonUtil.encode(msg.structuredData)));
sb.append(String.format(", \"timestamp\": %d", msg.timestamp.getEpochSecond()));
sb.append(String.format(", \"level\": %d", msg.severity.toNumber()));
sb.append(String.format(", \"_facility\": \"%s\"", msg.facility));
sb.append(String.format(", \"_severity\": \"%s\"", msg.severity));
sb.append(String.format(", \"_application\": \"%s\"", msg.application));
if(msg.processId != null) { sb.append(String.format(", \"_process-id\": \"%s\"", msg.processId)); }
if(msg.messageId != null) { sb.append(String.format(", \"_message-id\": \"%s\"", msg.messageId)); }
if(msg.structuredData != null) { sb.append(String.format(", \"_structured-data\": \"%s\"", JsonUtil.encode(msg.structuredData))); }
sb.append("}");
return sb.toString();
}
/**
* Return a Loki JSON formatted string of the SyslogMessage.
* https://grafana.com/docs/loki/latest/api/
* @param msg
* @return
*/
public static String toLoki(SyslogMessage msg) {
StringBuilder sb = new StringBuilder("{ \"streams\": [ { \"stream\": {");
sb.append(String.format(" \"hostname\": \"%s\",", msg.hostname));
sb.append(String.format(" \"facility\": \"%s\",", msg.facility));
sb.append(String.format(" \"level\": \"%s\",", msg.severity));
sb.append(String.format(" \"application\": \"%s\"", msg.application));
sb.append("}, \"values\": [ ");
sb.append(String.format("[ \"%d\", \"%s\" ]", msg.timestamp.getEpochSecond() * 1000000000L, getMessageLine(msg)));
sb.append(" ] } ] }");
log.debug(sb.toString());
return sb.toString();
}
private static String getMessageLine(SyslogMessage msg) {
StringBuilder sb = new StringBuilder();
sb.append(String.format("[%s.%s] ", msg.facility, msg.severity));
sb.append(String.format("%s ", msg.hostname));
sb.append(String.format("%s ", msg.application));
sb.append(JsonUtil.encode(msg.message));
return sb.toString();
}
static private String getPri(Facility facility, Severity severity) {
int pri = (facility.toNumber() * 8) + severity.toNumber();
return String.format("%c%d%c", '<', pri, '>');
}
}

View File

@ -1,101 +0,0 @@
/*
Copyright 2020 mark.nellemann@gmail.com
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package biz.nellemann.syslogd;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import picocli.CommandLine;
import picocli.CommandLine.Command;
import java.io.IOException;
import java.util.concurrent.Callable;
@Command(name = "syslogd",
mixinStandardHelpOptions = true,
description = "Basic syslog server.",
versionProvider = biz.nellemann.syslogd.VersionProvider.class)
public class SyslogServer implements Callable<Integer>, LogListener {
private final static Logger log = LoggerFactory.getLogger(SyslogServer.class);
@CommandLine.Option(names = {"-p", "--port"}, description = "Listening port [default: 514].")
private int port = 514;
@CommandLine.Option(names = "--no-udp", negatable = true, description = "Listen on UDP [default: true].")
boolean udpServer = true;
@CommandLine.Option(names = "--no-tcp", negatable = true, description = "Listen on TCP [default: true].")
boolean tcpServer = true;
@CommandLine.Option(names = "--no-ansi", negatable = true, description = "Output ANSI colors [default: true].")
boolean ansiOutput = true;
@CommandLine.Option(names = "--rfc5424", description = "Parse RFC-5424 messages [default: RFC-3164].")
boolean rfc5424 = false;
public static void main(String... args) {
int exitCode = new CommandLine(new SyslogServer()).execute(args);
System.exit(exitCode);
}
@Override
public Integer call() throws IOException {
if(udpServer) {
UdpServer udpServer = new UdpServer(port);
udpServer.addEventListener(this);
udpServer.start();
}
if(tcpServer) {
TcpServer tcpServer = new TcpServer(port);
tcpServer.addEventListener(this);
tcpServer.start();
}
return 0;
}
@Override
public void onLogEvent(LogEvent event) {
// Parse message
String message = event.getMessage();
SyslogMessage msg = null;
try {
if(rfc5424) {
msg = SyslogParser.parseRfc5424(message);
} else {
msg = SyslogParser.parseRfc3164(message);
}
} catch(Exception e) {
log.error("Problem parsing message: ", e);
}
if(msg != null) {
if(ansiOutput) {
System.out.println(msg.toAnsiString());
} else {
System.out.println(msg);
}
}
}
}

View File

@ -1,80 +0,0 @@
/*
Copyright 2020 mark.nellemann@gmail.com
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package biz.nellemann.syslogd;
import java.io.IOException;
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
public class UdpServer extends Thread {
protected DatagramSocket socket = null;
protected boolean listen = true;
public UdpServer() throws IOException {
this(514);
}
public UdpServer(int port) throws IOException {
super("SyslogServer");
socket = new DatagramSocket(port);
}
public void run() {
byte[] buf = new byte[4096];
DatagramPacket packet = new DatagramPacket(buf, buf.length);
while (listen) {
try {
socket.receive(packet);
String packetData = new String(packet.getData(), packet.getOffset(), packet.getLength(), "UTF8");
sendEvent(packetData);
} catch (Exception e) {
e.printStackTrace();
listen = false;
}
}
socket.close();
}
private synchronized void sendEvent(String message) {
LogEvent event = new LogEvent( this, message);
Iterator listeners = eventListeners.iterator();
while( listeners.hasNext() ) {
( (LogListener) listeners.next() ).onLogEvent( event );
}
}
/**
* Event Listener Configuration
*/
protected List<LogListener> eventListeners = new ArrayList<>();
public synchronized void addEventListener( LogListener l ) {
eventListeners.add( l );
}
public synchronized void removeEventListener( LogListener l ) {
eventListeners.remove( l );
}
}

View File

@ -1,3 +1,18 @@
/*
Copyright 2021 mark.nellemann@gmail.com
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package biz.nellemann.syslogd;
import picocli.CommandLine;

View File

@ -1,4 +1,19 @@
package biz.nellemann.syslogd;
/*
Copyright 2020 mark.nellemann@gmail.com
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package biz.nellemann.syslogd.msg;
import java.util.HashMap;
import java.util.Map;
@ -31,30 +46,30 @@ import java.util.Map;
*/
public enum Facility {
KERNEL(0),
USER(1),
MAIL(2),
DAEMON(3),
AUTH(4),
SYSLOG(5),
PRINT(6),
NEWS(7),
UUCP(8),
CRON(9),
AUTHPRIV(10),
FTP(11),
NTP(12),
AUDIT(13),
ALERT(14),
TIME(15),
LOCAL0(16),
LOCAL1(17),
LOCAL2(18),
LOCAL3(19),
LOCAL4(20),
LOCAL5(21),
LOCAL6(22),
LOCAL7(23);
kernel(0),
user(1),
mail(2),
daemon(3),
auth(4),
syslog(5),
print(6),
news(7),
uucp(8),
cron(9),
authpriv(10),
ftp(11),
ntp(12),
audit(13),
alert(14),
time(15),
local0(16),
local1(17),
local2(18),
local3(19),
local4(20),
local5(21),
local6(22),
local7(23);
// Cache lookups
private static final Map<Integer, Facility> BY_NUMBER = new HashMap<>();
@ -72,7 +87,7 @@ public enum Facility {
return this.facilityNumber;
}
private Integer facilityNumber;
private final Integer facilityNumber;
Facility(int facilityNumber) {
this.facilityNumber = facilityNumber;
}

View File

@ -1,4 +1,19 @@
package biz.nellemann.syslogd;
/*
Copyright 2020 mark.nellemann@gmail.com
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package biz.nellemann.syslogd.msg;
import java.util.HashMap;
import java.util.Map;
@ -15,14 +30,14 @@ import java.util.Map;
*/
public enum Severity {
EMERG(0),
ALERT(1),
CRIT(2),
ERROR(3),
WARN(4),
NOTICE(5),
INFO(6),
DEBUG(7);
emerg(0),
alert(1),
crit(2),
error(3),
warn(4),
notice(5),
info(6),
debug(7);
// Cache lookups
private static final Map<Integer, Severity> BY_NUMBER = new HashMap<>();
@ -40,7 +55,7 @@ public enum Severity {
return this.severityNumber;
}
private Integer severityNumber;
private final Integer severityNumber;
Severity(int severityNumber) {
this.severityNumber = severityNumber;
}

View File

@ -13,75 +13,58 @@
See the License for the specific language governing permissions and
limitations under the License.
*/
package biz.nellemann.syslogd;
package biz.nellemann.syslogd.msg;
import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonIgnore;
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import com.fasterxml.jackson.annotation.JsonProperty;
import java.time.Instant;
@JsonIgnoreProperties(ignoreUnknown = true)
public class SyslogMessage {
protected Facility facility;
protected Severity severity;
@JsonIgnore
public Facility facility = Facility.user;
@JsonProperty("level")
public Severity severity = Severity.info;
// The VERSION field denotes the version of the syslog protocol specification.
protected Integer version;
public String version;
// The TIMESTAMP field is a formalized timestamp derived from [RFC3339].
protected Instant timestamp;
@JsonProperty("timestamp") // 1670357783.694 - in GELF: seconds since UNIX epoch with optional decimal places for milliseconds
public Instant timestamp = Instant.now();
// The HOSTNAME field identifies the machine that originally sent the syslog message.
protected String hostname;
@JsonProperty("host")
public String hostname;
// The APP-NAME field SHOULD identify the device or application that originated the message.
protected String application;
@JsonProperty("_logger_name")
public String application;
// The PROCID field is often used to provide the process name or process ID associated with a syslog system.
protected String processId;
@JsonProperty("_thread_name")
public String processId;
// The MSGID SHOULD identify the type of message.
protected String messageId;
@JsonIgnore
public String messageId;
// STRUCTURED-DATA provides a mechanism to express information in a well defined, easily parseable and interpretable data format.
protected String structuredData;
@JsonProperty("full_message")
public String structuredData;
// The MSG part contains a free-form message that provides information about the event.
protected final String message;
@JsonProperty("short_message")
public String message;
SyslogMessage(final String message) {
@JsonCreator
public SyslogMessage(@JsonProperty("short_message") final String message) {
this.message = message;
}
public String toString() {
StringBuilder sb = new StringBuilder();
sb.append(timestamp.toString());
sb.append(String.format(" [%8.8s.%-6.6s] ", facility, severity));
sb.append(String.format(" %-16.16s ", hostname));
sb.append(String.format(" %-32.32s ", application));
sb.append(message);
return sb.toString();
}
public String toAnsiString() {
StringBuilder sb = new StringBuilder();
sb.append(timestamp.toString());
if(severity.toNumber() < 3 ) {
sb.append(Ansi.RED);
} else if(severity.toNumber() < 5) {
sb.append(Ansi.YELLOW);
} else {
sb.append(Ansi.GREEN);
}
sb.append(String.format(" [%8.8s.%-6.6s] ", facility, severity)).append(Ansi.RESET);
sb.append(Ansi.BLUE).append(String.format(" %-16.16s ", hostname)).append(Ansi.RESET);
sb.append(Ansi.CYAN).append(String.format(" %-32.32s ", application)).append(Ansi.RESET);
sb.append(message);
return sb.toString();
}
}

View File

@ -0,0 +1,29 @@
package biz.nellemann.syslogd.net;
import java.net.InetSocketAddress;
import java.net.SocketException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import biz.nellemann.syslogd.LogForwardEvent;
import biz.nellemann.syslogd.SyslogPrinter;
public class GelfClient extends UdpClient {
private final static Logger log = LoggerFactory.getLogger(GelfClient.class);
public GelfClient(InetSocketAddress inetSocketAddress) throws SocketException {
super(inetSocketAddress);
}
@Override
public void onForwardEvent(LogForwardEvent event) {
try {
send(SyslogPrinter.toGelf(event.getMessage()));
} catch (Exception e) {
log.warn("onForwardEvent() error", e);
}
}
}

View File

@ -0,0 +1,103 @@
/*
Copyright 2021 mark.nellemann@gmail.com
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package biz.nellemann.syslogd.net;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
import java.nio.charset.StandardCharsets;
import java.util.concurrent.ArrayBlockingQueue;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import biz.nellemann.syslogd.LogForwardEvent;
import biz.nellemann.syslogd.LogForwardListener;
import biz.nellemann.syslogd.SyslogPrinter;
public class LokiClient implements LogForwardListener, Runnable {
private final static Logger log = LoggerFactory.getLogger(LokiClient.class);
private final ArrayBlockingQueue<String> blockingQueue = new ArrayBlockingQueue<>(1024);
private final URL url;
public LokiClient(URL url) {
this.url = url;
}
public void send(String msg) throws MalformedURLException {
URL pushUrl = new URL(url, "/loki/api/v1/push");
HttpURLConnection con = null;
try {
con = (HttpURLConnection)pushUrl.openConnection();
con.setRequestMethod("POST");
con.setRequestProperty("Content-Type", "application/json");
con.setConnectTimeout(500);
con.setReadTimeout(150);
con.setDoOutput(true);
byte[] input = msg.getBytes(StandardCharsets.UTF_8);
try(OutputStream os = con.getOutputStream()) {
os.write(input, 0, input.length);
} catch (IOException e) {
log.warn(e.getMessage());
}
int responseCode = con.getResponseCode();
try (InputStream ignored = con.getInputStream()) {
if(responseCode != 204) {
log.warn("send() - response: " + responseCode);
}
}
} catch (IOException e) {
log.error("send() - error: " + e.getMessage());
} finally {
if(con != null) {
con.disconnect();
}
}
}
@Override
public void run() {
while (true) {
try {
send(blockingQueue.take());
} catch (MalformedURLException | InterruptedException e) {
log.warn(e.getMessage());
}
}
}
@Override
public void onForwardEvent(LogForwardEvent event) {
blockingQueue.offer(SyslogPrinter.toLoki(event.getMessage()));
}
}

View File

@ -13,27 +13,33 @@
See the License for the specific language governing permissions and
limitations under the License.
*/
package biz.nellemann.syslogd;
package biz.nellemann.syslogd.net;
import biz.nellemann.syslogd.LogReceiveEvent;
import biz.nellemann.syslogd.LogReceiveListener;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.DatagramPacket;
import java.net.ServerSocket;
import java.net.Socket;
import java.util.ArrayList;
import java.util.Iterator;
import java.nio.charset.StandardCharsets;
import java.util.List;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.ArrayList;
public class TcpServer {
private int port;
private final static Logger log = LoggerFactory.getLogger(TcpServer.class);
private final int port;
private ServerSocket serverSocket;
TcpServer() {
this(514);
}
TcpServer(int port) {
public TcpServer(int port) {
this.port = port;
}
@ -52,13 +58,17 @@ public class TcpServer {
* Event Listener Configuration
*/
protected List<LogListener> eventListeners = new ArrayList<>();
protected final List<LogReceiveListener> eventListeners = new ArrayList<>();
public synchronized void addEventListener( LogListener l ) {
eventListeners.add( l );
public synchronized void addEventListener(LogReceiveListener listener ) {
eventListeners.add( listener );
}
public synchronized void removeEventListener( LogListener l ) {
public synchronized void addEventListener(List<LogReceiveListener> listeners ) {
eventListeners.addAll(listeners);
}
public synchronized void removeEventListener( LogReceiveListener l ) {
eventListeners.remove( l );
}
@ -66,18 +76,20 @@ public class TcpServer {
private static class ClientHandler extends Thread {
protected List<LogListener> eventListeners;
protected final List<LogReceiveListener> eventListeners;
private Socket clientSocket;
private final Socket clientSocket;
private BufferedReader in;
public ClientHandler(Socket socket, List<LogListener> eventListeners) {
public ClientHandler(Socket socket, List<LogReceiveListener> eventListeners) {
this.clientSocket = socket;
this.eventListeners = eventListeners;
}
public void run() {
// GELF TCP does not support compression due to the use of the null byte (\0) as frame delimiter.
// Is \0 also used as frame delimiter for regular syslog messages ?
try {
in = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
String inputLine;
@ -85,24 +97,24 @@ public class TcpServer {
sendEvent(inputLine);
}
} catch (IOException e) {
e.printStackTrace();
log.warn("run() - read error: {}", e.getMessage());
}
try {
in.close();
clientSocket.close();
} catch (IOException e) {
e.printStackTrace();
log.warn("run() - close error: {}", e.getMessage());
}
}
private synchronized void sendEvent(String message) {
LogEvent event = new LogEvent( this, message );
Iterator listeners = eventListeners.iterator();
while( listeners.hasNext() ) {
( (LogListener) listeners.next() ).onLogEvent( event );
DatagramPacket packet = new DatagramPacket(message.getBytes(StandardCharsets.UTF_8), message.length());
LogReceiveEvent event = new LogReceiveEvent( this, packet);
for (LogReceiveListener eventListener : eventListeners) {
eventListener.onLogEvent(event);
}
}

View File

@ -0,0 +1,64 @@
/*
Copyright 2021 mark.nellemann@gmail.com
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package biz.nellemann.syslogd.net;
import biz.nellemann.syslogd.LogForwardEvent;
import biz.nellemann.syslogd.LogForwardListener;
import biz.nellemann.syslogd.SyslogPrinter;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
import java.net.*;
import java.nio.charset.StandardCharsets;
public class UdpClient implements LogForwardListener {
private final static Logger log = LoggerFactory.getLogger(UdpClient.class);
private final InetSocketAddress inetSocketAddress;
private final DatagramSocket socket;
public UdpClient(InetSocketAddress inetSocketAddress) throws SocketException {
this.inetSocketAddress = inetSocketAddress;
this.socket = new DatagramSocket();
}
public void send(String msg) {
byte[] buf = msg.getBytes(StandardCharsets.US_ASCII);
DatagramPacket packet = new DatagramPacket(buf, buf.length, inetSocketAddress.getAddress(), inetSocketAddress.getPort());
try {
socket.send(packet);
} catch (IOException e) {
log.error("send() - Could not send packet: " + e.getMessage());
}
}
public void close() {
socket.close();
}
@Override
public void onForwardEvent(LogForwardEvent event) {
try {
send(SyslogPrinter.toRfc5424(event.getMessage()));
} catch (Exception e) {
log.warn("onForwardEvent() error", e);
}
}
}

View File

@ -0,0 +1,94 @@
/*
Copyright 2020 mark.nellemann@gmail.com
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package biz.nellemann.syslogd.net;
import java.io.IOException;
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.util.ArrayList;
import java.util.List;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import biz.nellemann.syslogd.LogReceiveEvent;
import biz.nellemann.syslogd.LogReceiveListener;
public class UdpServer extends Thread {
private final static Logger log = LoggerFactory.getLogger(UdpServer.class);
protected DatagramSocket socket;
protected boolean listen = true;
public UdpServer(int port) throws IOException {
socket = new DatagramSocket(port);
}
@Override
public void run() {
byte[] buf = new byte[8192];
DatagramPacket packet = new DatagramPacket(buf, buf.length);
while (listen) {
try {
socket.receive(packet);
//String packetData = new String(packet.getData(), packet.getOffset(), packet.getLength(), StandardCharsets.UTF_8);
sendEvent(packet);
} catch (IOException e) {
log.error("run() - error: {}", e.getMessage());
listen = false;
}
}
socket.close();
}
/*
private synchronized void sendEvent(String message) {
LogReceiveEvent event = new LogReceiveEvent( this, message);
for (LogReceiveListener eventListener : eventListeners) {
eventListener.onLogEvent(event);
}
}
*/
private synchronized void sendEvent(DatagramPacket packet) {
LogReceiveEvent event = new LogReceiveEvent( this, packet);
for (LogReceiveListener eventListener : eventListeners) {
eventListener.onLogEvent(event);
}
}
/**
* Event Listener Configuration
*/
protected List<LogReceiveListener> eventListeners = new ArrayList<>();
public synchronized void addEventListener(LogReceiveListener listener ) {
eventListeners.add( listener );
}
public synchronized void addEventListener(List<LogReceiveListener> listeners ) {
eventListeners.addAll(listeners);
}
public synchronized void removeEventListener( LogReceiveListener l ) {
eventListeners.remove( l );
}
}

View File

@ -0,0 +1,136 @@
package biz.nellemann.syslogd.parser;
import java.time.Instant;
import java.util.Arrays;
import java.util.TreeMap;
import org.apache.commons.collections4.map.PassiveExpiringMap;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule;
import biz.nellemann.syslogd.msg.SyslogMessage;
/*
For more information about the GELF format, visit: https://go2docs.graylog.org/5-0/getting_in_log_data/gelf.html
*/
public class GelfParser extends SyslogParser {
private final static Logger log = LoggerFactory.getLogger(GelfParser.class);
private final ObjectMapper objectMapper;
private final int expiryInMills = 10_000;
private final PassiveExpiringMap<Integer, TreeMap<Integer, byte[]>> expiringMap = new PassiveExpiringMap<>(expiryInMills);
public GelfParser() {
objectMapper = new ObjectMapper();
objectMapper.registerModule(new JavaTimeModule());
}
/*
Magic Bytes - 2 bytes: 0x1e 0x0f
Message ID - 8 bytes: Must be the same for every chunk of this message.
Identifies the whole message and is used to reassemble the chunks later.
Generate from millisecond timestamp + hostname, for example.
Sequence number - 1 byte: The sequence number of this chunk starts at 0 and is always less than the sequence count.
Sequence count - 1 byte: Total number of chunks this message has.
All chunks MUST arrive within 5 seconds or the server will discard all chunks that have arrived or are in the process of arriving.
A message MUST NOT consist of more than 128 chunks.
*/
private SyslogMessage parseChunked(byte[] input) {
if(input.length < 12) return null;
byte[] messageId = { input[2], input[3], input[4], input[5], input[6], input[7], input[8], input[9] };
byte seqNumber = input[10];
byte seqTotal = input[11];
byte[] payload = Arrays.copyOfRange(input, 12, input.length);
log.debug("parseChunked() - msgId: {}, seqNo: {}, seqTot: {}, payload: {}", messageId, seqNumber, seqTotal, byteArrayToString(payload));
// messageId byte[] to int
int id = 0;
for (byte b : messageId) {
id = (id << 8) + (b & 0xFF);
}
TreeMap<Integer, byte[]> integerTreeMap;
if(expiringMap.containsKey(id)) {
integerTreeMap = expiringMap.get(id);
} else {
integerTreeMap = new TreeMap<>();
}
integerTreeMap.put((int)seqNumber, payload);
expiringMap.put(id, integerTreeMap);
if(integerTreeMap.size() >= seqTotal) {
StringBuilder sb = new StringBuilder();
integerTreeMap.forEach( (i, p) -> {
sb.append(byteArrayToString(p));
});
return parse(sb.toString());
}
return null;
}
@Override
public SyslogMessage parse(String input) {
if(!input.startsWith("{")) return null; // Avoid trying to parse non-JSON content
SyslogMessage message = null;
try {
message = objectMapper.readValue(input, SyslogMessage.class);
} catch (JsonProcessingException e) {
log.debug("parse() - error: {}", e.getMessage());
}
return message;
}
/*
zlib signatures at offset 0
78 01 : No Compression (no preset dictionary)
78 5E : Best speed (no preset dictionary)
78 9C : Default Compression (no preset dictionary)
78 DA : Best Compression (no preset dictionary)
78 20 : No Compression (with preset dictionary)
78 7D : Best speed (with preset dictionary)
78 BB : Default Compression (with preset dictionary)
78 F9 : Best Compression (with preset dictionary)
gzip signature at offset 0
1F 8B : GZIP compressed
*/
@Override
public SyslogMessage parse(byte[] input) {
if(input.length < 8) return null; // TODO: Find proper minimum input length ?
// Compressed data: 0x78 0x9c
if(input[0] == (byte)0x78 && input[1] == (byte)0x9c) {
input = decompress(input);
}
// Magic Bytes: 0x1e 0x0f
if(input[0] == (byte)0x1e && input[1] == (byte)0x0f) {
return parseChunked(input);
}
return parse(byteArrayToString(input));
}
@Override
public Instant parseTimestamp(String dateString) {
return null;
}
}

View File

@ -0,0 +1,101 @@
/*
This code is from https://gist.github.com/jjfiv/2ac5c081e088779f49aa, which is BSD licensed:
http://lemurproject.org/galago-license
*/
package biz.nellemann.syslogd.parser;
public class JsonUtil {
public static String encode(String input) {
if(input == null) {
return "";
}
StringBuilder output = new StringBuilder();
for (int i = 0; i < input.length(); i++) {
char ch = input.charAt(i);
// let's not put any nulls in our strings
if ((int) ch == 0) {
continue;
}
if (ch == '\n') {
output.append("\\n");
} else if (ch == '\t') {
output.append("\\t");
} else if (ch == '\r') {
output.append("\\r");
} else if (ch == '\\') {
output.append("\\\\");
} else if (ch == '"') {
output.append("\\\"");
} else if (ch == '\b') {
output.append("\\b");
} else if (ch == '\f') {
output.append("\\f");
} else if ((int) ch > 127) {
output.append(String.format("\\u%04x", (int) ch));
} else {
output.append(ch);
}
}
return output.toString();
}
public static String decode(String input) {
StringBuilder builder = new StringBuilder();
int i = 0;
while (i < input.length()) {
char delimiter = input.charAt(i);
i++; // consume letter or backslash
if (delimiter == '\\' && i < input.length()) {
// consume first after backslash
char ch = input.charAt(i);
i++;
if (ch == '\\' || ch == '/' || ch == '"' || ch == '\'') {
builder.append(ch);
} else if (ch == 'n') builder.append('\n');
else if (ch == 'r') builder.append('\r');
else if (ch == 't') builder.append('\t');
else if (ch == 'b') builder.append('\b');
else if (ch == 'f') builder.append('\f');
else if (ch == 'u') {
StringBuilder hex = new StringBuilder();
// expect 4 digits
if (i + 4 > input.length()) {
throw new RuntimeException("Not enough unicode digits! ");
}
for (char x : input.substring(i, i + 4).toCharArray()) {
if (!Character.isLetterOrDigit(x)) {
throw new RuntimeException("Bad character in unicode escape.");
}
hex.append(Character.toLowerCase(x));
}
i += 4; // consume those four digits.
int code = Integer.parseInt(hex.toString(), 16);
builder.append((char) code);
} else {
throw new RuntimeException("Illegal escape sequence: \\" + ch);
}
} else { // it's not a backslash, or it's the last character.
builder.append(delimiter);
}
}
return builder.toString();
}
}

View File

@ -0,0 +1,89 @@
/*
Copyright 2020 mark.nellemann@gmail.com
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package biz.nellemann.syslogd.parser;
import java.nio.charset.StandardCharsets;
import java.time.Instant;
import java.util.zip.DataFormatException;
import java.util.zip.Inflater;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import biz.nellemann.syslogd.msg.SyslogMessage;
public abstract class SyslogParser {
private final static Logger log = LoggerFactory.getLogger(SyslogParser.class);
public abstract SyslogMessage parse(final String input);
public abstract SyslogMessage parse(final byte[] input);
public abstract Instant parseTimestamp(final String dateString);
/**
* Converts syslog PRI field into Facility.
*
* @param pri
* @return
*/
public int getFacility(String pri) {
int priority = Integer.parseInt(pri);
int facility = priority >> 3;
return facility;
}
/**
* Converts syslog PRI field into Severity.
*
* @param pri
* @return
*/
public int getSeverity(String pri) {
int priority = Integer.parseInt(pri);
int severity = priority & 0x07;
return severity;
}
protected String byteArrayToString(byte[] input) {
return new String(input, 0, input.length, StandardCharsets.UTF_8);
}
protected byte[] decompress(byte[] data) {
byte[] result = new byte[data.length * 2];
try {
// Decompress the bytes
Inflater decompressor = new Inflater();
decompressor.setInput(data, 0, data.length);
//byte[] result = new byte[data.length * 2];
decompressor.inflate(result);
decompressor.end();
// Decode the bytes into a String
//uncompressed = new String(result, 0, resultLength, StandardCharsets.UTF_8);
} catch (DataFormatException e) {
log.error("decompress() - error: {}", e.getMessage());
}
return result;
}
}

View File

@ -0,0 +1,112 @@
/*
Copyright 2020 mark.nellemann@gmail.com
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package biz.nellemann.syslogd.parser;
import java.time.Instant;
import java.time.OffsetDateTime;
import java.time.ZoneId;
import java.time.format.DateTimeFormatter;
import java.time.format.DateTimeParseException;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import biz.nellemann.syslogd.msg.Facility;
import biz.nellemann.syslogd.msg.Severity;
import biz.nellemann.syslogd.msg.SyslogMessage;
public class SyslogParserRfc3164 extends SyslogParser {
private final static Logger log = LoggerFactory.getLogger(SyslogParserRfc3164.class);
private final Pattern pattern = Pattern.compile("^<(\\d{1,3})>(\\D{3}\\s+\\d{1,2} \\d{2}:\\d{2}:\\d{2})\\s+(Message forwarded from \\S+:|\\S+:?)\\s+([^\\s:]+):?\\s+(.*)", Pattern.CASE_INSENSITIVE);
private final DateTimeFormatter dateTimeFormatter = DateTimeFormatter.ofPattern("yyyy MMM [ ]d HH:mm:ss").withZone(ZoneId.systemDefault());
/**
* Parses [rfc3164](https://tools.ietf.org/html/rfc3164) syslog messages.
*
* @param input
* @return
* @throws NumberFormatException
*/
@Override
public SyslogMessage parse(final String input) throws NumberFormatException {
log.debug("parseRfc3164() " + input);
Matcher matcher = pattern.matcher(input);
if(!matcher.find()) {
System.err.println("!" + input);
return null;
}
String pri = matcher.group(1);
String date = matcher.group(2);
String hostname = matcher.group(3);
String application = matcher.group(4);
String msg = matcher.group(5);
if(hostname.endsWith(":")) {
String[] tmp = hostname.split(" ");
hostname = tmp[tmp.length-1];
hostname = hostname.substring(0, hostname.length()-1);
}
Integer facility = getFacility(pri);
Integer severity = getSeverity(pri);
SyslogMessage syslogMessage = new SyslogMessage(msg.trim());
syslogMessage.facility = Facility.getByNumber(facility);
syslogMessage.severity = Severity.getByNumber(severity);
syslogMessage.timestamp = parseTimestamp(date);
syslogMessage.hostname = hostname;
syslogMessage.application = application;
return syslogMessage;
}
@Override
public SyslogMessage parse(byte[] input) {
return parse(byteArrayToString(input));
}
/**
* Parse rfc3164 TIMESTAMP field into Instant.
*
* @param dateString
* @return
*/
@Override
public Instant parseTimestamp(String dateString) {
// We need to add current year to parse date correctly
OffsetDateTime odt = OffsetDateTime.now();
// Date: Mmm dd hh:mm:ss
try {
return Instant.from(dateTimeFormatter.parse(odt.getYear() + " " + dateString));
} catch(DateTimeParseException e) {
log.debug("parseTimestamp()", e);
}
log.warn("parseTimestamp() - Could not parse timestamp: " + dateString);
return Instant.now();
}
}

View File

@ -0,0 +1,137 @@
/*
Copyright 2020 mark.nellemann@gmail.com
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package biz.nellemann.syslogd.parser;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.time.Instant;
import java.util.Arrays;
import java.util.List;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import biz.nellemann.syslogd.msg.Facility;
import biz.nellemann.syslogd.msg.Severity;
import biz.nellemann.syslogd.msg.SyslogMessage;
public class SyslogParserRfc5424 extends SyslogParser {
private final static Logger log = LoggerFactory.getLogger(SyslogParserRfc5424.class);
private final Pattern pattern = Pattern.compile("^<(\\d{1,3})>(\\d+)\\s+(\\S+)\\s+(\\S+)\\s+(\\S+)\\s+(\\S+)\\s+(\\S+)\\s+(\\[.*\\]|-)\\s+(.*)", Pattern.CASE_INSENSITIVE);
/**
* Parses [rfc5424](https://tools.ietf.org/html/rfc5424) syslog messages.
*
* @param input
* @return
* @throws NumberFormatException
*/
@Override
public SyslogMessage parse(final String input) throws NumberFormatException {
log.debug("parseRfc5424() " + input);
Matcher matcher = pattern.matcher(input);
boolean matchFound = matcher.find();
if(!matchFound) {
log.debug("parseRfc5424() - Match not found in: " + input);
System.err.println("!" + input);
return null;
}
String pri = matcher.group(1);
String ver = matcher.group(2);
String date = matcher.group(3);
String host = matcher.group(4);
String app = matcher.group(5);
String procId = matcher.group(6);
String msgId = matcher.group(7);
String data = matcher.group(8);
String msg = matcher.group(9);
Integer facility = getFacility(pri);
Integer severity = getSeverity(pri);
SyslogMessage syslogMessage = new SyslogMessage(msg.trim());
syslogMessage.facility = Facility.getByNumber(facility);
syslogMessage.severity = Severity.getByNumber(severity);
syslogMessage.version = ver;
syslogMessage.timestamp = parseTimestamp(date);
syslogMessage.hostname = host;
if(app != null && !app.equals("-"))
syslogMessage.application = app;
if(procId != null && !procId.equals("-"))
syslogMessage.processId = procId;
if(msgId != null && !msgId.equals("-"))
syslogMessage.messageId = msgId;
if(data != null && !data.equals("-"))
syslogMessage.structuredData = data;
return syslogMessage;
}
@Override
public SyslogMessage parse(byte[] input) {
return parse(byteArrayToString(input));
}
/**
* Parse rfc5424 TIMESTAMP field into Instant.
*
* @param dateString
* @return
*/
@Override
public Instant parseTimestamp(String dateString) {
/*
https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html
ex1: 1985-04-12T23:20:50.52Z
ex2: 1985-04-12T19:20:50.52-04:00
ex3: 2003-10-11T22:14:15.003Z
ex4: 2003-08-24T05:14:15.000003-07:00
ex5: 2003-08-24T05:14:15.000000003-07:00
*/
List<String> formatStrings = Arrays.asList(
//"yyyy-MM-dd'T'HH:mm:ss.SS'X'",
"yyyy-MM-dd'T'HH:mm:ss.SSX",
"yyyy-MM-dd'T'HH:mm:ss.SSSX",
"yyyy-MM-dd'T'HH:mm:ss.SSSSSSX",
"yyyy-MM-dd'T'HH:mm:ss.SSSSSSSSSX"
);
for(String formatString : formatStrings)
{
try {
return new SimpleDateFormat(formatString).parse(dateString).toInstant();
}
catch (ParseException e) {
log.debug("parseTimestamp() " + e.getMessage());
}
}
log.warn("parseTimestamp() - Could not parse timestamp: " + dateString);
return Instant.now();
}
}

View File

@ -0,0 +1,4 @@
org.slf4j.simpleLogger.logFile=System.err
org.slf4j.simpleLogger.showDateTime=false
org.slf4j.simpleLogger.showShortLogName=true
org.slf4j.simpleLogger.levelInBrackets=true

View File

@ -0,0 +1,100 @@
package biz.nellemann.syslogd
import biz.nellemann.syslogd.msg.Facility
import biz.nellemann.syslogd.msg.Severity
import biz.nellemann.syslogd.msg.SyslogMessage
import biz.nellemann.syslogd.parser.GelfParser
import biz.nellemann.syslogd.parser.SyslogParser
import spock.lang.Specification
class GelfParserTest extends Specification {
SyslogParser syslogParser;
void setup() {
syslogParser = new GelfParser();
}
void "uncompressed GELF message"() {
setup:
def input = '{"version":"1.1","host":"pop-os.localdomain","short_message":"main() - Starting VTD-Camera","full_message":"main() - Starting VTD-Camera\\n","timestamp":1670357783.694,"level":4,"_thread_name":"main","_logger_name":"vtd.camera.Application"}'
when:
SyslogMessage msg = syslogParser.parse(input)
then:
msg.version == "1.1"
msg.message == "main() - Starting VTD-Camera"
msg.hostname == "pop-os.localdomain"
msg.application == "vtd.camera.Application"
msg.processId == "main"
msg.timestamp.toString() == "2022-12-06T20:16:23.694Z"
}
void "compressed GELF message"() {
setup:
byte[] input = [ 120, -100, -51, 81, 77, 107, -61, 48, 12, -3, 43, -63, -25, -38, -115, -77, -75, -51, 2, -127, -11, -80, -61, 96, -73, 94, 11, -63, 56, 90, -30, -59, 31, -63, -106, -53, -54, -40, 127, -97, 92, 86, -40, 79, -40, 69, -78, -97, -11, -12, -98, -28, 47, 118, -127, -104, 76, -16, -84, 99, 82, 72, -74, 97, 115, 72, 72, 23, -77, 114, 89, 115, 73, 65, -14, 102, 87, 11, -56, 92, -125, -57, -88, 44, -105, 66, 7, -73, 102, 4, 97, 60, 66, -12, -54, 18, 45, -51, 33, -30, -32, 32, 37, 53, 1, -15, -31, 66, -43, -3, 49, -29, 76, -39, 104, -123, -92, 113, -54, 90, 83, -63, 75, 121, -86, 114, 42, 84, 7, -67, 83, 113, 17, 30, -84, 5, -89, -68, 127, -98, -100, 50, -74, 40, 84, 8, 94, 81, -113, -90, -118, -32, 2, -62, 113, 28, 35, -79, 123, -39, -18, -124, 108, -91, 104, -102, 7, 33, -27, -95, 74, 4, 82, -13, -41, -79, -9, -39, 22, 43, -17, -108, -2, -127, -109, -77, 39, 47, 104, -56, 8, 42, -73, -78, 78, -18, 15, -11, -82, -106, -121, -89, 86, -44, -113, -5, 13, -77, 100, -52, -78, -114, 78, -125, 90, -41, 65, -121, 76, -21, -67, -110, -31, 113, 97, -65, 88, 113, 69, -128, -93, 61, -57, -126, -39, 48, 77, 16, -17, -80, -6, 8, 57, -118, 99, -119, 39, -48, 57, 26, -68, -2, -99, -21, -51, 36, -14, 13, 55, 34, 77, 72, -1, 60, -28, 72, -126, 108, 70, 92, 83, 119, -34, -98, -73, -29, 34, 110, -67, 5, -119, -35, 53, -63, 95, -88, 102, -115, 97, 44, 8, -50, 17, -44, 120, 87, 44, 76, -18, 77, -32, 109, -35, -42, 28, 62, 65, -13, -122, 125, -1, 0, -40, 60, -57, -72 ];
when:
SyslogMessage msg = syslogParser.parse(input)
then:
msg.version == "1.1"
msg.facility == Facility.user;
msg.severity == Severity.info;
msg.message == "event=AuthenticationSuccessEvent username=mark.nellemann@gmail.com tenant=2 remoteAddress=185.181.223.117 sessionId=null"
msg.hostname == "ip-10-1-101-250.eu-central-1.compute.internal"
msg.application == "ajour.AjourSecuritySuccessEventListener"
msg.processId == "http-nio-8080-exec-2"
msg.timestamp.toString() == "2022-12-08T12:16:38.046Z"
}
void "chunked GELF message"() {
setup:
byte[] chunk1 = [30, 15, -103, 51, -96, -10, -51, -31, 39, -41, 0, 2, 123, 34, 118, 101, 114, 115, 105, 111, 110, 34, 58, 34, 49, 46, 49, 34, 44, 34, 104, 111, 115, 116, 34, 58, 34, 105, 112, 45, 49, 48, 45, 49, 45, 49, 48, 49, 45, 49, 52, 46, 101, 117, 45, 99, 101, 110, 116, 114, 97, 108, 45, 49, 46, 99, 111, 109, 112, 117, 116, 101, 46, 105, 110, 116, 101, 114, 110, 97, 108, 34, 44, 34, 115, 104, 111, 114, 116, 95, 109, 101, 115, 115, 97, 103, 101, 34, 58, 34, 101, 118, 101, 110, 116, 61, 65, 117, 116, 104, 101, 110, 116, 105, 99, 97, 116, 105, 111, 110, 83, 117, 99, 99, 101, 115, 115, 69, 118, 101, 110, 116, 32, 117, 115, 101, 114, 110, 97, 109, 101, 61, 109, 97, 114, 107, 46, 110, 101, 108, 108, 101, 109, 97, 110, 110, 64, 103, 109, 97, 105, 108, 46, 99, 111, 109, 32, 116, 101, 110, 97, 110, 116, 61, 50, 32, 114, 101, 109, 111, 116, 101, 65, 100, 100, 114, 101, 115, 115, 61, 49, 56, 53, 46, 49, 56, 49, 46, 50, 50, 51, 46, 49, 49, 55, 32, 115, 101, 115, 115, 105, 111, 110, 73, 100, 61, 110, 117, 108, 108, 34, 44, 34, 102, 117, 108, 108, 95, 109, 101, 115, 115, 97, 103, 101, 34, 58, 34, 101, 118, 101, 110, 116, 61, 65, 117, 116, 104, 101, 110, 116, 105, 99, 97, 116, 105, 111, 110, 83, 117, 99, 99, 101, 115, 115, 69, 118, 101, 110, 116, 32, 117, 115, 101, 114, 110, 97, 109, 101, 61, 109, 97, 114, 107, 46, 110, 101, 108, 108, 101, 109, 97, 110, 110, 64, 103, 109, 97, 105, 108, 46, 99, 111, 109, 32, 116, 101, 110, 97, 110, 116, 61, 50, 32, 114, 101, 109, 111, 116, 101, 65, 100, 100, 114, 101, 115, 115, 61, 49, 56, 53, 46, 49, 56, 49, 46, 50, 50, 51, 46, 49, 49, 55, 32, 115, 101, 115, 115, 105, 111, 110, 73, 100, 61, 110, 117, 108, 108, 92, 110, 34, 44, 34, 116, 105, 109, 101, 115, 116, 97, 109, 112, 34, 58, 49, 54, 55, 49, 48, 48, 48, 53, 53, 54, 46, 56, 48, 50, 44, 34, 108, 101, 118, 101, 108, 34, 58, 54, 44, 34, 95, 97, 112, 112, 95, 99, 111, 117, 110, 116, 114, 121, 34, 58, 34, 100, 107, 34, 44, 34, 95, 97, 112, 112, 95, 110, 97, 109, 101, 34, 58, 34, 109, 105, 110, 116, 114, 34, 44, 34, 95, 108, 111, 103, 103, 101, 114, 95, 110, 97, 109, 101, 34, 58, 34, 97, 106, 111, 117, 114, 46, 65, 106, 111, 117, 114, 83, 101, 99, 117, 114, 105, 116, 121, 83, 117, 99, 99, 101, 115, 115, 69, 118, 101, 110, 116, 76, 105, 115, 116, 101, 110, 101, 114, 34, 44, 34, 95, 115, 101, 114, 118, 101, 114, 95, 117, 114, 108]
byte[] chunk2 = [30, 15, -103, 51, -96, -10, -51, -31, 39, -41, 1, 2, 34, 58, 34, 104, 116, 116, 112, 115, 58, 92, 47, 92, 47, 100, 107, 46, 109, 105, 110, 116, 114, 46, 97, 112, 112, 34, 44, 34, 95, 97, 112, 112, 95, 101, 110, 118, 34, 58, 34, 112, 114, 111, 100, 34, 44, 34, 95, 116, 104, 114, 101, 97, 100, 95, 110, 97, 109, 101, 34, 58, 34, 104, 116, 116, 112, 45, 110, 105, 111, 45, 56, 48, 56, 48, 45, 101, 120, 101, 99, 45, 52, 34, 125]
when:
syslogParser.parse(chunk1)
SyslogMessage msg = syslogParser.parse(chunk2)
then:
msg.message == "event=AuthenticationSuccessEvent username=mark.nellemann@gmail.com tenant=2 remoteAddress=185.181.223.117 sessionId=null"
}
void "chunked GELF unordered message"() {
setup:
byte[] chunk1 = [30, 15, -103, 51, -96, -10, -51, -31, 39, -41, 0, 2, 123, 34, 118, 101, 114, 115, 105, 111, 110, 34, 58, 34, 49, 46, 49, 34, 44, 34, 104, 111, 115, 116, 34, 58, 34, 105, 112, 45, 49, 48, 45, 49, 45, 49, 48, 49, 45, 49, 52, 46, 101, 117, 45, 99, 101, 110, 116, 114, 97, 108, 45, 49, 46, 99, 111, 109, 112, 117, 116, 101, 46, 105, 110, 116, 101, 114, 110, 97, 108, 34, 44, 34, 115, 104, 111, 114, 116, 95, 109, 101, 115, 115, 97, 103, 101, 34, 58, 34, 101, 118, 101, 110, 116, 61, 65, 117, 116, 104, 101, 110, 116, 105, 99, 97, 116, 105, 111, 110, 83, 117, 99, 99, 101, 115, 115, 69, 118, 101, 110, 116, 32, 117, 115, 101, 114, 110, 97, 109, 101, 61, 109, 97, 114, 107, 46, 110, 101, 108, 108, 101, 109, 97, 110, 110, 64, 103, 109, 97, 105, 108, 46, 99, 111, 109, 32, 116, 101, 110, 97, 110, 116, 61, 50, 32, 114, 101, 109, 111, 116, 101, 65, 100, 100, 114, 101, 115, 115, 61, 49, 56, 53, 46, 49, 56, 49, 46, 50, 50, 51, 46, 49, 49, 55, 32, 115, 101, 115, 115, 105, 111, 110, 73, 100, 61, 110, 117, 108, 108, 34, 44, 34, 102, 117, 108, 108, 95, 109, 101, 115, 115, 97, 103, 101, 34, 58, 34, 101, 118, 101, 110, 116, 61, 65, 117, 116, 104, 101, 110, 116, 105, 99, 97, 116, 105, 111, 110, 83, 117, 99, 99, 101, 115, 115, 69, 118, 101, 110, 116, 32, 117, 115, 101, 114, 110, 97, 109, 101, 61, 109, 97, 114, 107, 46, 110, 101, 108, 108, 101, 109, 97, 110, 110, 64, 103, 109, 97, 105, 108, 46, 99, 111, 109, 32, 116, 101, 110, 97, 110, 116, 61, 50, 32, 114, 101, 109, 111, 116, 101, 65, 100, 100, 114, 101, 115, 115, 61, 49, 56, 53, 46, 49, 56, 49, 46, 50, 50, 51, 46, 49, 49, 55, 32, 115, 101, 115, 115, 105, 111, 110, 73, 100, 61, 110, 117, 108, 108, 92, 110, 34, 44, 34, 116, 105, 109, 101, 115, 116, 97, 109, 112, 34, 58, 49, 54, 55, 49, 48, 48, 48, 53, 53, 54, 46, 56, 48, 50, 44, 34, 108, 101, 118, 101, 108, 34, 58, 54, 44, 34, 95, 97, 112, 112, 95, 99, 111, 117, 110, 116, 114, 121, 34, 58, 34, 100, 107, 34, 44, 34, 95, 97, 112, 112, 95, 110, 97, 109, 101, 34, 58, 34, 109, 105, 110, 116, 114, 34, 44, 34, 95, 108, 111, 103, 103, 101, 114, 95, 110, 97, 109, 101, 34, 58, 34, 97, 106, 111, 117, 114, 46, 65, 106, 111, 117, 114, 83, 101, 99, 117, 114, 105, 116, 121, 83, 117, 99, 99, 101, 115, 115, 69, 118, 101, 110, 116, 76, 105, 115, 116, 101, 110, 101, 114, 34, 44, 34, 95, 115, 101, 114, 118, 101, 114, 95, 117, 114, 108]
byte[] chunk2 = [30, 15, -103, 51, -96, -10, -51, -31, 39, -41, 1, 2, 34, 58, 34, 104, 116, 116, 112, 115, 58, 92, 47, 92, 47, 100, 107, 46, 109, 105, 110, 116, 114, 46, 97, 112, 112, 34, 44, 34, 95, 97, 112, 112, 95, 101, 110, 118, 34, 58, 34, 112, 114, 111, 100, 34, 44, 34, 95, 116, 104, 114, 101, 97, 100, 95, 110, 97, 109, 101, 34, 58, 34, 104, 116, 116, 112, 45, 110, 105, 111, 45, 56, 48, 56, 48, 45, 101, 120, 101, 99, 45, 52, 34, 125]
when:
syslogParser.parse(chunk2)
SyslogMessage msg = syslogParser.parse(chunk1)
then:
msg.message == "event=AuthenticationSuccessEvent username=mark.nellemann@gmail.com tenant=2 remoteAddress=185.181.223.117 sessionId=null"
}
void "junk GET request"() {
setup:
def input = 'GET /'
when:
SyslogMessage msg = syslogParser.parse(input)
then:
msg == null
}
}

View File

@ -0,0 +1,52 @@
package biz.nellemann.syslogd
import biz.nellemann.syslogd.parser.JsonUtil
import spock.lang.Specification
class JsonUtilTest extends Specification {
def "test short decode"() {
setup:
def testShort = 'Eating a piece of \u03c0 (pi)'
when:
def result = JsonUtil.decode(testShort)
then:
result == 'Eating a piece of π (pi)'
}
def "test long decode"() {
setup:
def testLong = 'I stole this guy from wikipedia: \ud83d\ude02' // emoji "face with tears of joy"
when:
def result = JsonUtil.decode(testLong)
then:
result == 'I stole this guy from wikipedia: 😂'
}
def "test quotes decode"() {
setup:
def testQuote = 'here it comes \" to wreck the day...'
when:
def result = JsonUtil.decode(testQuote)
then:
result == 'here it comes " to wreck the day...'
}
def "test newline decode"() {
setup:
def testQuote = 'here it comes \n to wreck the day...'
when:
def result = JsonUtil.decode(testQuote)
then:
result == 'here it comes \n to wreck the day...'
}
}

View File

@ -0,0 +1,116 @@
package biz.nellemann.syslogd
import biz.nellemann.syslogd.msg.SyslogMessage
import biz.nellemann.syslogd.parser.SyslogParser
import biz.nellemann.syslogd.parser.SyslogParserRfc3164
import spock.lang.Specification
import java.text.DateFormat
import java.time.Instant
import java.time.OffsetDateTime
import java.time.ZoneId
import java.time.format.DateTimeFormatter;
class SyslogParserRfc3164Test extends Specification {
SyslogParser syslogParser;
void setup() {
syslogParser = new SyslogParserRfc3164();
}
void "test rfc3164 aix/vios message"() {
setup:
def input = "<13>Sep 23 08:37:09 Message forwarded from p924vio1: padmin: test"
when:
SyslogMessage msg = syslogParser.parse(input)
then:
msg.message == "test"
msg.hostname == "p924vio1"
msg.application == "padmin"
}
void "test another rfc3164 aix/vios message"() {
setup:
def input = "<13>Dec 18 10:09:22 Message forwarded from p924vio1: root: [errnotify] seq: 24266 - AA8AB241 1218100920 T O OPERATOR OPERATOR NOTIFICATION"
when:
SyslogMessage msg = syslogParser.parse(input)
then:
msg.message == "[errnotify] seq: 24266 - AA8AB241 1218100920 T O OPERATOR OPERATOR NOTIFICATION"
msg.hostname == "p924vio1"
msg.application == "root"
}
void "test rfc3164 normal message"() {
setup:
def input = "<13>Sep 23 08:53:28 xps13 mark: adfdfdf3432434 abcdefghijklmnopqrstuvwxyz"
when:
SyslogMessage msg = syslogParser.parse(input)
then:
msg.message == "adfdfdf3432434 abcdefghijklmnopqrstuvwxyz"
msg.hostname == "xps13"
msg.application == "mark"
}
void "test rsyslogd sudo message"() {
setup:
String input = "<85>Oct 5 17:13:41 xps13 sudo: mark : TTY=pts/1 ; PWD=/etc/rsyslog.d ; USER=root ; COMMAND=/usr/sbin/service rsyslog restart"
when:
SyslogMessage msg = syslogParser.parse(input)
then:
msg.application == "sudo"
msg.message == "mark : TTY=pts/1 ; PWD=/etc/rsyslog.d ; USER=root ; COMMAND=/usr/sbin/service rsyslog restart"
}
void "test gdm-session message"() {
setup:
String input = "<12>Oct 5 18:31:01 xps13 /usr/lib/gdm3/gdm-x-session[1921]: (EE) event5 - CUST0001:00 06CB:76AF Touchpad: kernel bug: Touch jump detected and discarded."
when:
SyslogMessage msg = syslogParser.parse(input)
then:
msg.application == "/usr/lib/gdm3/gdm-x-session[1921]"
msg.message == "(EE) event5 - CUST0001:00 06CB:76AF Touchpad: kernel bug: Touch jump detected and discarded."
}
void "test intellij message"() {
setup:
String input = "<14>Oct 6 05:10:26 xps13 com.jetbrains.IntelliJ-IDEA-Ulti git4idea.commands.GitStandardProgressAnalyzer\$1.onLineAvailable(GitStandardProgressAnalyzer.java:45)"
when:
SyslogMessage msg = syslogParser.parse(input)
then:
msg.application == "com.jetbrains.IntelliJ-IDEA-Ulti"
msg.message == "git4idea.commands.GitStandardProgressAnalyzer\$1.onLineAvailable(GitStandardProgressAnalyzer.java:45)"
}
void "test parseRfc3164Timestamp"() {
setup:
OffsetDateTime odt = OffsetDateTime.now();
String dateString = "Sep 12 20:50:13"
when:
Instant instant = syslogParser.parseTimestamp(dateString)
DateTimeFormatter formatter = DateTimeFormatter.ofPattern("YYYY MMM dd HH:mm:ss").withZone(ZoneId.systemDefault());
then:
assert formatter.format(instant).equals(odt.getYear() + " " + dateString);
}
}

View File

@ -0,0 +1,123 @@
package biz.nellemann.syslogd
import biz.nellemann.syslogd.msg.SyslogMessage
import biz.nellemann.syslogd.parser.SyslogParser
import biz.nellemann.syslogd.parser.SyslogParserRfc5424
import spock.lang.Specification
import java.time.Instant
class SyslogParserRfc5424Test extends Specification {
SyslogParser syslogParser;
void setup() {
syslogParser = new SyslogParserRfc5424();
}
void "test rfc5424 message"() {
setup:
def input = '<13>1 2020-09-23T08:57:30.950699+02:00 xps13 mark - - [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"] adfdfdf3432434565656'
when:
SyslogMessage msg = syslogParser.parse(input)
then:
msg.message == "adfdfdf3432434565656"
msg.structuredData == "[exampleSDID@32473 iut=\"3\" eventSource=\"Application\" eventID=\"1011\"]"
}
void "test rfc5424 example message"() {
setup:
def input = "<34>1 2003-10-11T22:14:15.003Z mymachine.example.com su - ID47 - BOM'su root' failed for lonvick on /dev/pts/8"
when:
SyslogMessage msg = syslogParser.parse(input)
then:
msg.hostname == "mymachine.example.com"
msg.application == "su"
msg.messageId == "ID47"
msg.processId == null
msg.message == "BOM'su root' failed for lonvick on /dev/pts/8"
}
void "test rfc5424 example2 message"() {
setup:
def input = "<165>1 2003-08-24T05:14:15.000003-07:00 192.0.2.1 myproc 8710 - - %% It's time to make the do-nuts."
when:
SyslogMessage msg = syslogParser.parse(input)
then:
msg.hostname == "192.0.2.1"
msg.application == "myproc"
msg.processId == "8710"
msg.structuredData == null
}
void "test parseRfc5424Timestamp ex1"() {
setup:
String dateString = "1985-04-12T23:20:50.52Z"
when:
Instant inst = syslogParser.parseTimestamp(dateString)
then:
inst.toEpochMilli() == 482196050052
inst.toString() == "1985-04-12T23:20:50.052Z"
}
void "test parseRfc5424Timestamp ex2"() {
setup:
String dateString = "1985-04-12T19:20:50.52-04:00"
when:
Instant inst = syslogParser.parseTimestamp(dateString)
then:
inst.toEpochMilli() == 482196050052
inst.toString() == "1985-04-12T23:20:50.052Z"
}
void "test parseRfc5424Timestamp ex3"() {
setup:
String dateString = "2003-10-11T22:14:15.003Z"
when:
Instant inst = syslogParser.parseTimestamp(dateString)
then:
inst.toEpochMilli() == 1065910455003
inst.toString() == "2003-10-11T22:14:15.003Z"
}
void "test parseRfc5424Timestamp ex4"() {
setup:
String dateString = "2003-08-24T05:14:15.000003-07:00"
when:
Instant inst = syslogParser.parseTimestamp(dateString)
then:
inst.toEpochMilli() == 1061727255003
inst.toString() == "2003-08-24T12:14:15.003Z"
}
void "test parseRfc5424Timestamp ex5"() {
setup:
String dateString = "2003-08-24T05:14:15.000000003-07:00"
when:
Instant inst = syslogParser.parseTimestamp(dateString)
then:
inst.toEpochMilli() == 1061727255003
inst.toString() == "2003-08-24T12:14:15.003Z"
}
}

View File

@ -1,111 +1,33 @@
package biz.nellemann.syslogd
import biz.nellemann.syslogd.msg.Facility
import biz.nellemann.syslogd.msg.Severity
import biz.nellemann.syslogd.parser.SyslogParser
import biz.nellemann.syslogd.parser.SyslogParserRfc5424
import spock.lang.Specification
import java.time.Instant
import java.time.OffsetDateTime;
class SyslogParserTest extends Specification {
void "test rfc5424 message"() {
SyslogParser syslogParser;
setup:
def input = "<13>1 2020-09-23T08:57:30.950699+02:00 xps13 mark - - [timeQuality tzKnown=\"1\" isSynced=\"1\" syncAccuracy=\"125500\"] adfdfdf3432434565656"
when:
SyslogMessage msg = SyslogParser.parseRfc5424(input)
then:
msg.message == "adfdfdf3432434565656"
void setup() {
syslogParser = new SyslogParserRfc5424();
}
void "test rfc3164 aix/vios message"() {
setup:
def input = "<13>Sep 23 08:37:09 Message forwarded from p924vio1: padmin: test"
void "test facility LOCAL0"() {
when:
SyslogMessage msg = SyslogParser.parseRfc3164(input)
int code = syslogParser.getFacility("132")
then:
msg.message == "test"
msg.hostname == "p924vio1"
msg.application == "padmin"
code == Facility.local0.toNumber()
}
void "test rfc3164 normal message"() {
setup:
def input = "<13>Sep 23 08:53:28 xps13 mark: adfdfdf3432434"
void "test severity WARN"() {
when:
SyslogMessage msg = SyslogParser.parseRfc3164(input)
int code = syslogParser.getSeverity("132")
then:
msg.message == "adfdfdf3432434"
msg.hostname == "xps13"
msg.application == "mark"
}
void "test rsyslogd sudo message"() {
setup:
String input = "<85>Oct 5 17:13:41 xps13 sudo: mark : TTY=pts/1 ; PWD=/etc/rsyslog.d ; USER=root ; COMMAND=/usr/sbin/service rsyslog restart"
when:
SyslogMessage msg = SyslogParser.parseRfc3164(input)
then:
msg.application == "sudo"
msg.message == "mark : TTY=pts/1 ; PWD=/etc/rsyslog.d ; USER=root ; COMMAND=/usr/sbin/service rsyslog restart"
}
void "test gdm-session message"() {
setup:
String input = "<12>Oct 5 18:31:01 xps13 /usr/lib/gdm3/gdm-x-session[1921]: (EE) event5 - CUST0001:00 06CB:76AF Touchpad: kernel bug: Touch jump detected and discarded."
when:
SyslogMessage msg = SyslogParser.parseRfc3164(input)
then:
msg.application == "/usr/lib/gdm3/gdm-x-session[1921]"
msg.message == "(EE) event5 - CUST0001:00 06CB:76AF Touchpad: kernel bug: Touch jump detected and discarded."
}
void "test intellij message"() {
setup:
String input = "<14>Oct 6 05:10:26 xps13 com.jetbrains.IntelliJ-IDEA-Ulti git4idea.commands.GitStandardProgressAnalyzer\$1.onLineAvailable(GitStandardProgressAnalyzer.java:45)"
when:
SyslogMessage msg = SyslogParser.parseRfc3164(input)
then:
msg.application == "com.jetbrains.IntelliJ-IDEA-Ulti"
msg.message == "git4idea.commands.GitStandardProgressAnalyzer\$1.onLineAvailable(GitStandardProgressAnalyzer.java:45)"
}
void "test parseRfc3164Timestamp"() {
setup:
OffsetDateTime odt = OffsetDateTime.now()
String dateString = "Sep 12 20:50:13"
when:
Instant inst = SyslogParser.parseRfc3164Timestamp(dateString)
then:
inst.toString() == "${odt.getYear()}-09-12T20:50:13Z"
}
void "test parseRfc5424Timestamp"() {
setup:
String dateString = "2020-09-22T20:10:30.925438+02:00"
when:
Instant inst = SyslogParser.parseRfc5424Timestamp(dateString)
then:
inst.toString() == "2020-09-22T18:10:30.925438Z"
inst.toEpochMilli() == 1600798230925l
code == Severity.warn.toNumber()
}
}

View File

@ -0,0 +1,54 @@
package biz.nellemann.syslogd
import biz.nellemann.syslogd.msg.SyslogMessage
import biz.nellemann.syslogd.parser.SyslogParser
import biz.nellemann.syslogd.parser.SyslogParserRfc5424
import spock.lang.Specification
class SyslogPrinterTest extends Specification {
void setup() {
}
void "to plain"() {
setup:
SyslogParser syslogParser = new SyslogParserRfc5424();
String input = '<13>1 2020-09-23T08:57:30.950699+02:00 xps13 mark - - [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"] adfdfdf3432434565656 abcdefghijklmnopqrstuvwxyz'
SyslogMessage msg = syslogParser.parse(input)
when:
String output = SyslogPrinter.toString(msg)
then:
output.endsWith("abcdefghijklmnopqrstuvwxyz")
}
void "test toGelf"() {
setup:
SyslogParser syslogParser = new SyslogParserRfc5424();
String input = '<13>1 2020-09-23T08:57:30.950699+02:00 xps13 mark - - [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"] adfdfdf3432434565656 abcdefghijklmnopqrstuvwxyz'
SyslogMessage msg = syslogParser.parse(input)
when:
String output = SyslogPrinter.toGelf(msg)
then:
output.contains("_structured-data")
}
void "test toLoki"() {
setup:
SyslogParser syslogParser = new SyslogParserRfc5424();
String input = '<13>1 2020-09-23T08:57:30.950699+02:00 xps13 mark - - [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"] adfdfdf3432434565656 abcdefghijklmnopqrstuvwxyz'
SyslogMessage msg = syslogParser.parse(input)
when:
String output = SyslogPrinter.toLoki(msg)
then:
output == '{ "streams": [ { "stream": { "hostname": "xps13", "facility": "user", "level": "notice", "application": "mark"}, "values": [ [ "1600845200000000000", "[user.notice] xps13 mark adfdfdf3432434565656 abcdefghijklmnopqrstuvwxyz" ] ] } ] }'
}
}

View File

@ -0,0 +1,6 @@
org.slf4j.simpleLogger.logFile=System.err
org.slf4j.simpleLogger.showDateTime=false
org.slf4j.simpleLogger.showShortLogName=true
org.slf4j.simpleLogger.dateTimeFormat=yyyy-MM-dd HH:mm:ss.SSS
org.slf4j.simpleLogger.levelInBrackets=true
org.slf4j.simpleLogger.defaultLogLevel=debug