nellemann/svci
1
0
Fork 0
Code Issues Pull requests Packages Projects Releases Wiki Activity

Support for InfluxDB 2.x (now requires 1.8 or later) #2

Merged
nellemann merged 3 commits from influxdb2 into main 2023-05-25 13:23:01 +00:00
Conversation 0 Commits 3 Files changed 9 +50 -183
11 changed files with 50 additions and 183 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files
Showing only changes of commit 88e45c21d9 - Show all commits

27
README.md
View file

@ -1,4 +1,4 @@
# Spectrum Virtualize Insights
# Spectrum Virtualize Insights / SVCi
**SVCi** is a utility that collects metrics from one or more *IBM SAN Volume Controllers*. The metric data is processed and saved into an InfluxDB time-series database. Grafana is used to visualize the metrics data from InfluxDB through provided dashboards, or your own customized dashboards.
@ -13,6 +13,13 @@ Some of my other related projects are:
- [syslogd](https://git.data.coop/nellemann/syslogd) for redirecting syslog and GELF to remote logging destinations
## Screenshots
![screenshot](doc/screenshots/v7000-8_4_2_0-1.png)
More screenshots can be found in the [doc/screenshots/](doc/screenshots) folder.
## Installation and Setup
There are few steps in the installation.
@ -30,8 +37,6 @@ There are few steps in the installation.
Install InfluxDB (v. **1.8** or later) on a host which is network accessible by the SVCi utility (the default InfluxDB port is 8086). You can install Grafana on the same server or any server which are able to connect to the InfluxDB database. The Grafana installation needs to be accessible from your browser (default on port 3000). The default settings for both InfluxDB and Grafana will work fine as a start.
- You can download [Grafana ppc64le](https://www.power-devops.com/grafana) and [InfluxDB ppc64le](https://www.power-devops.com/influxdb) packages for most Linux distributions and AIX on the [Power DevOps](https://www.power-devops.com/) site.
- Binaries for amd64/x86 are available from the [Grafana website](https://grafana.com/grafana/download) (select the **OSS variant**) and [InfluxDB website](https://portal.influxdata.com/downloads/) and most likely directly from your Linux distributions repositories.
- Create the empty *svci* database by running the **influx** CLI command and type:
```text
@ -65,16 +70,6 @@ Install *SVCi* on a host, that can connect to your SAN Volume Controllers (on po
This is most likely due to timezone, date and/or NTP not being configured correctly on the SAN Volune Controller and/or host running SVCi.
### Start InfluxDB and Grafana at boot (systemd compatible Linux)
```shell
systemctl enable influxdb
systemctl start influxdb
systemctl enable grafana-server
systemctl start grafana-server
```
### InfluxDB Retention Policy
Examples for changing the default InfluxDB retention policy for the svci database:
@ -103,7 +98,6 @@ systemctl restart svci
journalctl -f -u svci # to check log output
```
### AIX Notes
To install (or upgrade) on AIX, you need to pass the *--ignoreos* flag to the *rpm* command:
@ -118,10 +112,6 @@ rpm -Uvh --ignoreos svci-x.y.z-n.noarch.rpm
Screenshots of the provided Grafana dashboard can be found in the [doc/screenshots/](doc/screenshots) folder.
## Known problems
## Development Information
You need Java (JDK) version 8 or later to build hmci.
@ -175,3 +165,4 @@ Setup Grafana to connect to the InfluxDB container by defining a new datasource
If you are [connecting](https://docs.influxdata.com/influxdb/v2.7/tools/grafana/) to InfluxDB v2.x, then add a custom http header, enter bucket as database and disable authorization.
- Authorization = Token abcdef_random_token_from_nfluxdb==
- Import dashboards from the [doc/dashboards/](doc/dashboards/) folder.

4
doc/dashboards/SVCi - IBM Spectrum Virtualize.json
View file

@ -76,7 +76,7 @@
}
]
},
"description": "https://git.data.coop/nellemann/svci",
"description": "https://git.data.coop/nellemann/svci/ - Metrics collected from IBM Spectrum Virtualize.",
"editable": true,
"fiscalYearStartMonth": 0,
"graphTooltip": 0,
@ -97,7 +97,7 @@
},
"id": 16,
"options": {
"content": "## Metrics collected from IBM Spectrum Virtualize\n \nFor more information: [https://git.data.coop/nellemann/svci](https://git.data.coop/nellemann/svci)\n ",
"content": "## Metrics collected from IBM Spectrum Virtualize\n \nFor more information visit: [git.data.coop/nellemann/svci](https://git.data.coop/nellemann/svci)\n ",
"mode": "markdown"
},
"pluginVersion": "9.1.6",

4
doc/readme-aix.md
View file

@ -2,8 +2,6 @@
Please note that the software versions referenced in this document might have changed and might not be available/working unless updated.
More details are available in the [README.md](../README.md) file.
- Grafana and InfluxDB can be downloaded from the [Power DevOps](https://www.power-devops.com/) website - look under the *Monitor* section.
- Ensure Java (version 8 or later) is installed and available in your PATH.
@ -14,7 +12,7 @@ More details are available in the [README.md](../README.md) file.
[Download](https://git.data.coop/nellemann/-/packages/generic/svci/) the latest version of SVCi packaged for rpm.
```shell
rpm -ivh --ignoreos svci-0.0.1-1_all.rpm
rpm -ivh --ignoreos svci-0.0.3-1_all.rpm
cp /opt/svci/doc/svci.toml /etc/
```

19
doc/readme-debian.md
View file

@ -1,15 +1,13 @@
# Instruction for Debian / Ubuntu Systems
# Instructions for Debian / Ubuntu Systems
Please note that the software versions referenced in this document might have changed and might not be available/working unless updated.
More details are available in the [README.md](../README.md) file.
All commands should be run as root or through sudo.
## Install the Java Runtime from repository
```shell
apt-get install default-jre-headless
apt-get install default-jre-headless wget
```
@ -25,13 +23,17 @@ systemctl start influxdb
Run the ```influx``` cli command and create the *svci* database.
```sql
CREATE DATABASE "svci" WITH DURATION 365d REPLICATION 1;
```
## Download and Install Grafana
```shell
sudo apt-get install -y adduser libfontconfig1
wget https://dl.grafana.com/oss/release/grafana_9.1.3_amd64.deb
dpkg -i grafana_9.1.3_amd64.deb
apt-get install -y adduser libfontconfig1
wget https://dl.grafana.com/oss/release/grafana_9.1.7_amd64.deb
dpkg -i grafana_9.1.7_amd64.deb
systemctl daemon-reload
systemctl enable grafana-server
systemctl start grafana-server
@ -45,7 +47,8 @@ When logged in to Grafana (port 3000, admin/admin) create a datasource that poin
[Download](https://git.data.coop/nellemann/-/packages/generic/svci/) the latest version of SVCi packaged for deb.
```shell
dpkg -i svci_0.0.1-1_all.deb
wget https://git.data.coop/api/packages/nellemann/generic/svci/v0.0.3/svci_0.0.3-1_all.deb
dpkg -i svci_0.0.3-1_all.deb
cp /opt/svci/doc/svci.toml /etc/
cp /opt/svci/doc/svci.service /etc/systemd/system/
systemctl daemon-reload

19
doc/readme-redhat.md
View file

@ -2,16 +2,14 @@
Please note that the software versions referenced in this document might have changed and might not be available/working unless updated.
More details are available in the [README.md](../README.md) file. If you are running Linux on Power (ppc64le) you should look for ppc64le packages at the [Power DevOps](https://www.power-devops.com/) website.
All commands should be run as root or through sudo.
## Install the Java Runtime from repository
```shell
dnf install java-11-openjdk-headless
dnf install java-11-openjdk-headless wget
# or
yum install java-11-openjdk-headless
yum install java-11-openjdk-headless wget
```
@ -27,12 +25,15 @@ systemctl start influxdb
Run the ```influx``` cli command and create the *svci* database.
```sql
CREATE DATABASE "svci" WITH DURATION 365d REPLICATION 1;
```
## Download and Install Grafana
```shell
wget https://dl.grafana.com/oss/release/grafana-9.1.3-1.x86_64.rpm
rpm -ivh grafana-9.1.3-1.x86_64.rpm
wget https://dl.grafana.com/oss/release/grafana-9.1.7-1.x86_64.rpm
rpm -ivh grafana-9.1.7-1.x86_64.rpm
systemctl daemon-reload
systemctl enable grafana-server
systemctl start grafana-server
@ -43,15 +44,13 @@ When logged in to Grafana (port 3000, admin/admin) create a datasource that poin
## Download and Install svci
[Download](https://git.data.coop/nellemann/-/packages/generic/svci/) the latest version of SVCi packaged for rpm.
```shell
rpm -ivh svci-0.0.1-1_all.rpm
wget https://git.data.coop/api/packages/nellemann/generic/svci/v0.0.3/svci-0.0.3-1.noarch.rpm
rpm -ivh svci-0.0.3-1_all.rpm
cp /opt/svci/doc/svci.toml /etc/
cp /opt/svci/doc/svci.service /etc/systemd/system/
systemctl daemon-reload
systemctl enable svci
systemctl start svci
```
Now modify */etc/svci.toml* and test your setup by running ```/opt/svci/bin/svci -d``` manually and verify connection to SVC and InfluxDB. Afterwards start service with ```systemctl start svci``` .

12
doc/readme-suse.md
View file

@ -9,7 +9,7 @@ All commands should be run as root or through sudo.
## Install the Java Runtime from repository
```shell
zypper install java-11-openjdk-headless
zypper install java-11-openjdk-headless wget
```
@ -25,12 +25,15 @@ systemctl start influxdb
Run the ```influx``` cli command and create the *svci* database.
```sql
CREATE DATABASE "svci" WITH DURATION 365d REPLICATION 1;
```
## Download and Install Grafana
```shell
wget https://dl.grafana.com/oss/release/grafana-9.1.3-1.x86_64.rpm
rpm -ivh --nodeps grafana-9.1.3-1.x86_64.rpm
wget https://dl.grafana.com/oss/release/grafana-9.1.7-1.x86_64.rpm
rpm -ivh --nodeps grafana-9.1.7-1.x86_64.rpm
systemctl daemon-reload
systemctl enable grafana-server
systemctl start grafana-server
@ -44,7 +47,8 @@ When logged in to Grafana (port 3000, admin/admin) create a datasource that poin
[Download](https://git.data.coop/nellemann/-/packages/generic/svci/) the latest version of SVCi packaged for rpm.
```shell
rpm -ivh svci-0.0.1-1_all.rpm
wget https://git.data.coop/api/packages/nellemann/generic/svci/v0.0.3/svci-0.0.3-1.noarch.rpm
rpm -ivh svci-0.0.3-1_all.rpm
cp /opt/svci/doc/svci.toml /etc/
cp /opt/svci/doc/svci.service /etc/systemd/system/
systemctl daemon-reload

0
doc/screenshots/v7000-8_4_2_0.png → doc/screenshots/v7000-8_4_2_0-1.png
View file

Side by side

After

Width:  |  Height:  |  Size: 1.1 MiB

BIN
doc/screenshots/v7000-8_4_2_0-2.png Normal file
View file

Binary file not shown.
Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 180 KiB

After

Width:  |  Height:  |  Size: 180 KiB

14
doc/svci.toml
View file

@ -1,4 +1,5 @@
# SVCi Configuration
# Copy this file into /etc/svci.toml and customize it to your environment.
###
### Define one InfluxDB to save metrics into
@ -25,11 +26,16 @@ bucket = "svci"
### Each entry must be named [svc.<something-unique>]
###
# SVC on our primary site
###
### Define one or more SVC's to query for metrics
### Each entry must be named [svc.<something-unique>]
###
[svc.site1]
url = "https://10.10.10.12:7443"
url = "https://10.10.10.5:7443"
username = "superuser"
password = "password"
refresh = 30
trust = true # Ignore SSL cert. errors
refresh = 30 # How often to query SVC for data - in seconds
trust = true # Ignore SSL cert. errors (due to default self-signed cert.)

22
src/test/groovy/biz/nellemann/svci/InfluxClientTest.groovy
View file

@ -1,22 +0,0 @@
package biz.nellemann.svci
import biz.nellemann.svci.dto.toml.InfluxConfiguration
import spock.lang.Ignore
import spock.lang.Specification
@Ignore
class InfluxClientTest extends Specification {
InfluxClient influxClient
def setup() {
influxClient = new InfluxClient(new InfluxConfiguration("http://localhost:8086", "root", "", "svci"))
influxClient.login()
}
def cleanup() {
influxClient.logoff()
}
}

112
src/test/groovy/biz/nellemann/svci/VolumeControllerTest.groovy
View file

@ -1,112 +0,0 @@
package biz.nellemann.svci
import org.mockserver.integration.ClientAndServer
import org.mockserver.logging.MockServerLogger
import org.mockserver.socket.PortFactory
import org.mockserver.socket.tls.KeyStoreFactory
import spock.lang.Ignore
import spock.lang.Shared
import spock.lang.Specification
import javax.net.ssl.HttpsURLConnection
@Ignore
class VolumeControllerTest extends Specification {
@Shared
private static ClientAndServer mockServer;
@Shared
private RestClient serviceClient
@Shared
private VolumeController volumeController
@Shared
private File metricsFile
def setupSpec() {
HttpsURLConnection.setDefaultSSLSocketFactory(new KeyStoreFactory(new MockServerLogger()).sslContext().getSocketFactory());
mockServer = ClientAndServer.startClientAndServer(PortFactory.findFreePort());
serviceClient = new RestClient(String.format("http://localhost:%d", mockServer.getPort()), "user", "password", false)
MockResponses.prepareClientResponseForLogin(mockServer)
//MockResponses.prepareClientResponseForManagementConsole(mockServer)
//MockResponses.prepareClientResponseForManagedSystem(mockServer)
//MockResponses.prepareClientResponseForVirtualIOServer(mockServer)
//MockResponses.prepareClientResponseForLogicalPartition(mockServer)
serviceClient.login()
volumeController = new VolumeController(serviceClient, );
volumeController.discover()
}
def cleanupSpec() {
mockServer.stop()
}
def setup() {
}
def "test we got entry"() {
expect:
volumeController.entry.getName() == "Server-9009-42A-SN21F64EV"
}
void "test getDetails"() {
when:
volumeController.deserialize(metricsFile.getText('UTF-8'))
List<Measurement> listOfMeasurements = volumeController.getDetails()
then:
listOfMeasurements.size() == 1
listOfMeasurements.first().tags['servername'] == 'Server-9009-42A-SN21F64EV'
listOfMeasurements.first().fields['utilizedProcUnits'] == 0.00458
listOfMeasurements.first().fields['assignedMem'] == 40448.0
}
void "test getMemoryMetrics"() {
when:
volumeController.deserialize(metricsFile.getText('UTF-8'))
List<Measurement> listOfMeasurements = volumeController.getMemoryMetrics()
then:
listOfMeasurements.size() == 1
listOfMeasurements.first().fields['totalMem'] == 1048576.000
}
void "test getProcessorMetrics"() {
when:
volumeController.deserialize(metricsFile.getText('UTF-8'))
List<Measurement> listOfMeasurements = volumeController.getProcessorMetrics()
then:
listOfMeasurements.size() == 1
listOfMeasurements.first().fields['availableProcUnits'] == 4.65
}
void "test getSystemSharedProcessorPools"() {
when:
volumeController.deserialize(metricsFile.getText('UTF-8'))
List<Measurement> listOfMeasurements = volumeController.getSharedProcessorPools()
then:
listOfMeasurements.size() == 4
listOfMeasurements.first().fields['assignedProcUnits'] == 22.00013
}
void "test getPhysicalProcessorPool"() {
when:
volumeController.deserialize(metricsFile.getText('UTF-8'))
List<Measurement> listOfMeasurements = volumeController.getPhysicalProcessorPool()
then:
listOfMeasurements.size() == 1
listOfMeasurements.first().fields['assignedProcUnits'] == 22.0
}
}