Databases

Chronograf Tutorial: Dashboards and Alert Rules for InfluxDB

When a teammate opens Chronograf for the first time, they usually click the one host they care about and stop there. The pre-canned dashboard can’t say which core is pinned or whether the box stopped reporting, and nobody gets paged at 2 a.m.

Original content from computingforgeeks.com - post 188796

Chronograf fixes both from the same browser tab. Its dashboards are cells of InfluxQL or Flux queries tied together by template variables, and its alert rule builder writes a Kapacitor TICKscript for you every time you press Save. That means a rule you click together in the UI is a real Kapacitor task you can read, version and restore from the shell.

This Chronograf tutorial builds a custom dashboard with a variable and a Flux cell and moves it through JSON. Then it creates a relative CPU rule and a deadman rule and fires both on purpose. We ran every step in November 2026 on an Ubuntu 26.04 VM with Chronograf 1.11.7, InfluxDB 1.13.2, Kapacitor 1.8.9 and Telegraf 1.40.3 from the InfluxData repository.

Prerequisites

Everything here assumes a working TICK stack with Chronograf already connected to InfluxDB and Kapacitor. If you don’t have one yet, the TICK stack install on Ubuntu covers the repository, the four services, the Chronograf connection wizard and the firewall. Our lab host is called mon01 and has:

  • 2 vCPUs and one network interface named ens4
  • Telegraf writing its default system inputs to the telegraf database
  • Chronograf on port 8888 with source “InfluxDB mon01” and Kapacitor connection “Kapacitor mon01”
  • stress-ng (sudo apt install -y stress-ng), so we can push the CPU hard enough to trip an alert

Before building anything, confirm all four services are up and Telegraf is actually writing. A dashboard built against an empty database looks exactly like a broken query, so it’s cheaper to rule that out first:

systemctl is-active influxdb telegraf kapacitor chronograf
influx -execute "SHOW MEASUREMENTS" -database telegraf

On our test server all four report active, and the default Telegraf inputs show up as ten measurements:

active
active
active
active
name: measurements
name
----
cpu
disk
diskio
kernel
mem
net
netstat
processes
swap
system

If the query returns nothing or says the database was not found, fix the Telegraf output before going further, because every cell and every rule below reads from it.

Enabling Flux in InfluxDB for Chronograf

Chronograf’s cell editor has a Flux mode, but InfluxDB 1.x ships with the Flux endpoint switched off. Skip this step and the Flux button still appears, yet every script you run will come back with an error. The setting lives in the [http] section of /etc/influxdb/influxdb.conf as a commented line, so a single sed uncomments and flips it:

sudo sed -i "s/^  # flux-enabled = false/  flux-enabled = true/" /etc/influxdb/influxdb.conf
grep -n "^  flux-enabled" /etc/influxdb/influxdb.conf
sudo systemctl restart influxdb

The grep should print the edited line with its line number, which was 327 on our install:

327:  flux-enabled = true

Test the endpoint with a one-line Flux query over the HTTP API before you trust it from the UI. This asks for the last memory reading from the past minute:

curl -s -XPOST localhost:8086/api/v2/query -H "Accept:application/csv" -H "Content-type:application/vnd.flux" -d "from(bucket:\"telegraf/autogen\") |> range(start:-1m) |> filter(fn:(r) => r._measurement == \"mem\" and r._field == \"used_percent\") |> last()"

A CSV row with a _value means Flux is live. Note the bucket name: in InfluxDB 1.x, Flux addresses data as database/retention-policy, so telegraf/autogen is the bucket every Flux cell below uses.

,result,table,_start,_stop,_time,_value,_field,_measurement,host
,_result,0,2026-11-12T05:45:42.920002696Z,2026-11-12T05:46:42.920002696Z,2026-11-12T05:46:30Z,16.11039242776566,used_percent,mem,mon01

Memory sat at 16% on the idle VM, which is the baseline you’ll see in the first dashboard cell.

Reading the pre-canned host dashboards

Open Chronograf and click the eye icon for Host List. Chronograf looks at which measurements each host writes and matches them to its bundled “apps”, so mon01 shows its CPU, load and a system app link.

Chronograf Host List showing mon01 with CPU, load and the system app

Clicking the host name opens a pre-canned dashboard built from those measurements: CPU, load, memory, disk, network and processes.

Chronograf pre-canned system dashboard for mon01 built from Telegraf measurements

These dashboards are generated on the fly and cannot be edited or saved. That is the main reason to build your own. The pre-canned view averages every core into one line, and the moment you want a variable, a Flux query or a second host on the same screen, you need a dashboard you own. If you’re still choosing between Chronograf and other tools, our roundup of open source Linux monitoring tools compares the alternatives.

Building a custom Chronograf dashboard

Go to Dashboards in the sidebar and click Create Dashboard.

Chronograf Dashboards page with the Create Dashboard button marked

Chronograf creates an empty dashboard called “Name This Dashboard” and drops you on it. Click that title, type mon01 overview and press Enter. The dashboard is saved immediately; there’s no separate Save button at this level, only on each cell.

Adding an InfluxQL cell with the query builder

Click Add Data to open the cell editor. The query builder has three columns that you work through left to right, and Chronograf writes the InfluxQL as you click:

  1. DB.RetentionPolicy: pick telegraf.autogen
  2. Measurements & Tags: pick mem
  3. Fields: tick used_percent
Chronograf cell editor building an InfluxQL query for mem used_percent in telegraf.autogen
  1. 1Database
  2. 2Measurement
  3. 3Field

The graph fills in straight away with a mean("used_percent") query grouped by :interval:. Rename the cell by clicking its title at the top of the editor (“Memory used %” on ours) and press the green check mark to save it. That green check is the only save there is, so closing the editor any other way throws the cell away.

Creating a :cpu: template variable

A template variable turns one cell into a cell per core, host or interface, picked from a dropdown at the top of the dashboard. Click Variables in the dashboard header, then Add Template Variable. Name it cpu (Chronograf wraps it in colons for you) and set the type to Tag Values.

Creating the cpu template variable from Tag Values on the cpu measurement in Chronograf
  1. 1Name
  2. 2Type
  3. 3Create

The Tag Values type asks for a database, measurement and tag key. The defaults on our install were already telegraf, cpu and cpu, and the preview listed cpu-total, cpu0 and cpu1. Click Create. The other types in that dropdown are worth knowing, because they decide where the dropdown values come from:

  • Databases, Measurements, Field Keys, Tag Keys, Tag Values: filled by a built-in SHOW query
  • CSV and Map: values you type yourself, for a fixed list of hosts or friendly names
  • InfluxQL Meta Query and Flux Query: any query you write, for lists the built-ins can’t express
  • Text: a free text box, handy for a threshold value you want to tweak on the fly

The built-in types quote the value for you, so in a query you write "cpu" = :cpu: without adding quotes yourself. Add quotes anyway and the substituted value ends up wrapped in two sets of quotes, which breaks the query. The Chronograf template variable reference lists the quoting rule for each type.

Writing a raw InfluxQL query against the variable

Add a second cell with the Add Cell icon (a grid with a plus sign) in the dashboard header. This time skip the builder and type the query straight into the editor, because the builder can’t put a template variable into the WHERE clause:

SELECT mean("usage_user") AS "usage_user", mean("usage_system") AS "usage_system" FROM "telegraf"."autogen"."cpu" WHERE time > :dashboardTime: AND time < :upperDashboardTime: AND "cpu" = :cpu: GROUP BY time(:interval:) FILL(null)

Click Submit Query, then Show Template Values to check what InfluxDB will actually receive. The editor swaps to a read-only preview of the substituted values, with :cpu: replaced by the selected value (cpu-total here, the default) and the time variables replaced by real timestamps. It also disables editing until you click the button again, which caught us out the first time: you type, nothing happens, and the reason is a banner reading “Previewing substituted Template Variable values, editing is disabled”.

Raw InfluxQL cell query filtered by the cpu template variable with Show Template Values on
  1. 1Raw InfluxQL
  2. 2Substituted values

Name this cell “CPU by core” and save it. The three time variables are worth using in every raw query you write, because they tie the cell to the dashboard’s time picker:

  • :dashboardTime:: the start of the range in the time dropdown
  • :upperDashboardTime:: the end of the range, now() unless you used the date picker
  • :interval:: the GROUP BY time() bucket Chronograf picks for the cell’s width

Hard-code time > now() - 1h instead and the cell ignores the time picker, so it shows a different window from every other cell on the same dashboard.

Adding a Flux cell

Some things are awkward in InfluxQL and short in Flux. Network counters are the usual example: Telegraf writes bytes_recv as an ever-growing total, and what you want on a graph is bytes per second. Add a third cell, click the Flux toggle above the editor and paste this script:

from(bucket: "telegraf/autogen")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r._measurement == "net" and r._field == "bytes_recv" and r.interface == "ens4")
  |> derivative(unit: 1s, nonNegative: true)
  |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false)
  |> yield(name: "bytes_per_sec")

Click Run Script. The Flux variables do the same job as the InfluxQL ones: v.timeRangeStart and v.timeRangeStop follow the time picker and v.windowPeriod stands in for :interval:. The nonNegative: true flag matters, because a counter reset after a reboot would otherwise draw one huge negative spike. Replace ens4 with your interface name (ip -br link lists them), name the cell “ens4 receive rate (Flux)” and save it.

The Flux toggle, Run Script and the green check (Save cell) are the three controls to find in this view.

Chronograf cell editor in Flux mode running a derivative over net bytes_recv for ens4
  1. 1Flux
  2. 2Run Script
  3. 3Save cell

Back on the dashboard, the :cpu: dropdown now sits above the cells. Pick cpu1 and only the CPU cell changes, because it’s the only cell that references the variable. Switching between cpu0, cpu1 and cpu-total is how you spot one pinned core behind a calm average, which is exactly the detail the pre-canned dashboard hides.

Finished mon01 overview Chronograf dashboard with the cpu template variable dropdown

Every cell on this dashboard is yours to edit, resize or export, which the pre-canned view never allowed.

Sending a Data Explorer query to a dashboard

Data Explorer (the graph icon in the sidebar) is where you poke at data before deciding it deserves a cell. It has the same builder and editor as the cell editor, without the commitment. We picked telegraf.autogen, the system measurement and the load1 field to see the one-minute load average.

When a query is worth keeping, click Send to Dashboard at the top right. The dialog asks for the target dashboard (you can pick several, or “Send to a New Dashboard”) and a cell name. We sent it to mon01 overview as “Load average 1m”.

Chronograf Data Explorer Send to Dashboard dialog with target dashboard and cell name
  1. 1Target dashboard
  2. 2Cell name
  3. 3Send

Pick the target dashboard, type the cell name and press Send to Dashboard. The dashboards API confirms the cell landed:

curl -s http://localhost:8888/chronograf/v1/dashboards | jq -r '.dashboards[] | [.id, .name, (.cells | length)] | @tsv'

Dashboard 3 now has four cells, the three we built plus the Data Explorer one:

3	mon01 overview	4

That same /chronograf/v1/dashboards endpoint is what you’ll script backups against later, so it’s worth knowing it answers without a login on a default install. Keep port 8888 off the public internet until authentication is in place.

Exporting and importing dashboards as JSON

Dashboards live in Chronograf’s own BoltDB file, not in InfluxDB, so a rebuilt Chronograf server starts empty. Exporting to JSON is how you copy a dashboard to another Chronograf or keep it in Git, and it does not need any plugin. On the Dashboards page, hover over the dashboard’s row and click Export. The browser downloads a file named after the dashboard, mon01-overview.json in our case. Ours was 9,543 bytes, exported before the Data Explorer cell was added.

Exporting the mon01 overview Chronograf dashboard to JSON

The file wraps the dashboard in a meta block that records the Chronograf version and the source each cell was bound to. Reading it with jq shows why the import step asks about sources:

jq ".meta" mon01-overview.json

Source 1 is the InfluxDB connection on the server that exported it, which means nothing to a different Chronograf:

{
  "chronografVersion": "1.11.7",
  "sources": {
    "1": {
      "name": "InfluxDB mon01",
      "link": "/chronograf/v1/sources/1"
    }
  }
}

The template variable travels with the dashboard, query and all. This is the :cpu: definition straight from the exported file:

jq ".dashboard.templates[] | {tempVar, type, query}" mon01-overview.json

The Tag Values type is stored as a SHOW TAG VALUES template with the database, measurement and key filled in, so it rebuilds its dropdown from whatever data the new server has:

{
  "tempVar": ":cpu:",
  "type": "tagValues",
  "query": {
    "influxql": "SHOW TAG VALUES ON :database: FROM :measurement: WITH KEY=:tagKey:",
    "db": "telegraf",
    "measurement": "cpu",
    "tagKey": "cpu",
    "fieldKey": ""
  }
}

The cells are worth a look too, because they show which ones the builder still understands. Print each cell’s name, query type and the builder fields it saved:

jq -c '.dashboard.cells[] | {name, type: .queries[0].type, database: .queries[0].queryConfig.database, measurement: .queries[0].queryConfig.measurement}' mon01-overview.json

Only the memory cell, the one built with the query builder, carries a database and measurement:

{"name":"Memory used %","type":"influxql","database":"telegraf","measurement":"mem"}
{"name":"ens4 receive rate (Flux)","type":"flux","database":"","measurement":""}
{"name":"CPU by core","type":"influxql","database":"","measurement":""}

The Flux cell and the raw InfluxQL cell save their query as plain text, so their builder fields stay empty. If you script changes to an exported dashboard, that tells you where to look: a builder cell’s database lives in queryConfig, while a Flux or raw cell only has it inside the query string. A find and replace of the database name has to cover both.

To import, click Import Dashboard on the Dashboards page, drop the JSON file in the dialog and click Continue.

Chronograf Import Dashboard dialog with the exported JSON file loaded

A Reconcile Sources step follows, listing each source in the file with a dropdown to map it to a source on this Chronograf. “Dynamic Source” means the cell follows whichever source is selected in the UI, which is the safer choice when the target server’s IDs differ. For each row, the Map to source choice is the Available Sources dropdown: pick Dynamic Source, then click Done.

Chronograf Reconcile Sources step mapping the imported dashboard to Dynamic Source
  1. 1Map to source
  2. 2Done

One thing to expect: the import keeps the original name, so importing onto the same server gives you two dashboards called “mon01 overview” with nothing to tell them apart except the ID in the URL. Rename the copy right away, or delete it, before someone edits the wrong one.

Two mon01 overview dashboards in Chronograf after importing onto the same server

We deleted the imported copy through the dashboards API before moving on, which left dashboard 3 as the only one.

Building a relative alert rule in Chronograf

A static threshold such as “CPU idle below 10%” pages you for every nightly backup. A relative rule compares the current value with an earlier one, so it catches a sudden change and ignores a level that’s high but steady. Go to Alerting, then Alert Rules, then Build Alert Rule, and fill it in from top to bottom:

  1. Name: mon01 idle drop
  2. Alert Type: Relative
  3. Time Series: telegraf.autogen, then cpu, expand the cpu tag and tick cpu-total only, then the usage_idle field
  4. Conditions: send an alert when the change compared to previous 1m is less than -40
  5. Alert Handlers: add log with the file /var/log/kapacitor/chronograf-alerts.log

The three choices that make this rule are Relative, cpu-total only, and a change below -40.

Chronograf relative alert rule on cpu-total usage_idle with a change below -40
  1. 1Relative
  2. 2cpu-total only
  3. 3Change below -40

Filtering on cpu-total is the part people miss. The builder leaves Group By empty, so with the tag open every core and the total pour into one ungrouped stream, and the one-minute comparison ends up subtracting one core’s idle from another’s. The condition reads as “idle fell by more than 40 points within a minute”, which a two-core box only does when something takes over the CPU.

InfluxData’s guide to Chronograf alert rules covers the Threshold type we skip here. The handler list is long (Slack, PagerDuty, email, Telegram, Teams, OpsGenie, Alerta, exec, post and more), and each one needs its own Kapacitor config before it delivers anything. The log handler needs nothing, which makes it the right first handler: you prove the rule fires before you debug an SMTP relay. Below the handler, set the message:

{{ .ID }} is {{ .Level }} value: {{ index .Fields "value" }}

The template buttons under the message box insert the same fields, and “Send alert only when condition state changes” is ticked by default. Leave it ticked, or a sustained problem logs a line every ten seconds. Click Save Rule; Chronograf sends the TICKscript to Kapacitor and enables it in the same step.

Alert message template and state-change option on the Chronograf alert rule

The message template is stored with the rule, so the deadman rule in the next section can reuse the same log file and wording.

Adding a deadman rule for a silent host

Every threshold rule has a blind spot: if Telegraf stops, no data arrives, nothing crosses the threshold and the dashboard just stops drawing. A deadman rule alerts on the absence of data. Build a second rule named mon01 telegraf silent, choose Deadman, pick telegraf.autogen and the system measurement, and set “Send Alert if Data is missing for” to 1m. Then press Save Rule.

Chronograf deadman alert rule with a one minute missing-data period
  1. 1Deadman
  2. 2Missing for
  3. 3Save Rule

The Missing for field shows the period as 1m0s once picked. One minute is fine for a lab where you want to watch it fire; on a real fleet, 5m avoids paging for a Telegraf restart during patching. Add the same log handler and message before you save. The Alert Rules page now lists both rules with the blue Task Enabled dot, and the TICKscripts page lists the same two names as stream tasks.

Chronograf Alert Rules list with the relative and deadman rules enabled in Kapacitor

That list is the first place to check when an alert you expected never arrives, because a rule can be saved and still not be running.

Reading the TICKscript Chronograf generated

Each saved rule is an ordinary Kapacitor task with an ID starting chronograf-v1-. That matters when you hand alerting to someone who prefers the shell, because they can see and manage the same rules with the kapacitor CLI:

kapacitor list tasks

Both rules are enabled, executing and reading telegraf.autogen:

ID                                                 Type      Status    Executing Databases and Retention Policies
chronograf-v1-671ff584-1097-4ef9-b19a-f8e179befe02 stream    enabled   true      ["telegraf"."autogen"]
chronograf-v1-db61918a-04f0-49ac-b4b0-8a8c80328c69 stream    enabled   true      ["telegraf"."autogen"]

Show the relative rule’s script with kapacitor show and the task ID. The output starts with the task status and ends with a DOT graph of node statistics; the part worth reading is the TICKscript itself:

kapacitor show chronograf-v1-671ff584-1097-4ef9-b19a-f8e179befe02

Every field from the builder became a variable at the top, and the relative condition is a shift joined back to the live stream:

var db = 'telegraf'

var rp = 'autogen'

var measurement = 'cpu'

var groupBy = []

var whereFilter = lambda: ("cpu" == 'cpu-total') AND isPresent("usage_idle")

var name = 'mon01 idle drop'

var idVar = name

var message = '{{ .ID }} is {{ .Level }} value: {{ index .Fields "value" }}'

var idTag = 'alertID'

var levelTag = 'level'

var messageField = 'message'

var durationField = 'duration'

var outputDB = 'chronograf'

var outputRP = 'autogen'

var outputMeasurement = 'alerts'

var triggerType = 'relative'

var shift = 1m

var crit = -40

var data = stream
    |from()
        .database(db)
        .retentionPolicy(rp)
        .measurement(measurement)
        .groupBy(groupBy)
        .where(whereFilter)
    |eval(lambda: "usage_idle")
        .as('value')

var past = data
    |shift(shift)

var current = data

var trigger = past
    |join(current)
        .as('past', 'current')
    |eval(lambda: float("current.value" - "past.value"))
        .keep()
        .as('value')
    |alert()
        .crit(lambda: "value" < crit)
        .stateChangesOnly()
        .message(message)
        .id(idVar)
        .idTag(idTag)
        .levelTag(levelTag)
        .messageField(messageField)
        .durationField(durationField)
        .stateChangesOnly()
        .log('/var/log/kapacitor/chronograf-alerts.log')

trigger
    |eval(lambda: float("value"))
        .as('value')
        .keep()
    |influxDBOut()
        .create()
        .database(outputDB)
        .retentionPolicy(outputRP)
        .measurement(outputMeasurement)
        .tag('alertName', name)
        .tag('triggerType', triggerType)

trigger
    |httpOut('output')

Three details in there explain behaviour you’d otherwise puzzle over:

  • The cpu-total filter from the builder is the whereFilter lambda
  • Every alert is also written to the alerts measurement in the chronograf database, which is where Alert History reads from
  • .stateChangesOnly() appears twice; it’s harmless, but don’t copy it into hand-written scripts
Chronograf TICKscripts page listing the two generated Kapacitor tasks

Clicking a rule name on the TICKscripts page opens the same script in Chronograf’s TICKscript editor, which validates it as you type.

Chronograf TICKscript editor showing the generated relative rule as valid

The builder only understands the scripts it writes, so once you hand-edit one for logic the builder can’t express, keep editing it as TICKscript from then on. The next article in this series picks up from that point with hand-written Kapacitor TICKscript alerts.

Firing both rules and reading Alert History

A rule you have never seen fire is a guess. Trip the relative rule by loading both cores at once, which drops cpu-total idle from near 100% to near zero within one Telegraf interval. The trailing & keeps your shell free for the next steps while the three-minute run goes on:

nohup stress-ng --cpu 2 --timeout 180s >/dev/null 2>&1 &

Watch Alerting, then Alert History, or sudo tail -f /var/log/kapacitor/chronograf-alerts.log in a second shell. We waited for the idle drop to go CRITICAL and back to OK before we stopped Telegraf to trip the deadman:

sudo systemctl stop telegraf

Wait about two minutes, because a deadman checks once per period and can need up to two periods to notice, then start Telegraf again:

sudo systemctl start telegraf

The log handler writes one JSON object per alert. Pull out the time, level and message with jq rather than reading the raw lines, which carry the full alert details as an escaped string:

sudo jq -r '[.time, .level, .message] | @tsv' /var/log/kapacitor/chronograf-alerts.log

Both rules fired and both recovered:

2026-11-12T06:18:20Z	CRITICAL	mon01 idle drop is CRITICAL value: -57.570590221815735
2026-11-12T06:19:30Z	OK	mon01 idle drop is OK value: 0
2026-11-12T06:21:00Z	CRITICAL	mon01 telegraf silent is CRITICAL value: <no value>
2026-11-12T06:22:00Z	OK	mon01 telegraf silent is OK value: <no value>

The relative rule fired with idle down 57.6 points on the value a minute earlier. It cleared 70 seconds later, while the three-minute stress-ng run was still going, because idle had stopped falling compared with a minute before. That’s the trade-off of a relative rule: it tells you something changed, not that something is still wrong. The deadman fired on a minute boundary, at 06:21:00, and cleared on the next one after Telegraf came back.

Open Alerting, then Alert History, to see the same four events with a coloured level dot, the time in your browser’s timezone and the value.

Chronograf Alert History with CRITICAL and OK events for both rules

Our browser ran three hours ahead of UTC, so the 06:18:20 alert shows as 09:18:20 there. The Kapacitor side writes those rows into InfluxDB, and you can query them directly:

influx -database chronograf -precision rfc3339 -execute 'SELECT alertName, level, triggerType, value FROM alerts'

The deadman rows carry a value even though its log message didn’t:

name: alerts
time                 alertName             level    triggerType value
----                 ---------             -----    ----------- -----
2026-11-12T06:18:20Z mon01 idle drop       CRITICAL relative    -57.570590221815735
2026-11-12T06:19:30Z mon01 idle drop       OK       relative    0
2026-11-12T06:21:00Z mon01 telegraf silent CRITICAL deadman     0
2026-11-12T06:22:00Z mon01 telegraf silent OK       deadman     15

Because the history is just a measurement, it follows the retention policy of the chronograf database, which is infinite by default. A busy fleet piles up alert rows forever unless you give that database a shorter policy.

Deadman message shows “value: <no value>”

The deadman lines above end in <no value> because a deadman node doesn’t produce a value field. Its only field is emitted, the number of points it counted in the period, which the log confirms:

sudo jq -c 'select(.id=="mon01 telegraf silent") | {level, fields: .data.series[0].columns}' /var/log/kapacitor/chronograf-alerts.log

Only time and emitted are there:

{"level":"CRITICAL","fields":["time","emitted"]}
{"level":"OK","fields":["time","emitted"]}

Alert History shows a number anyway because the generated TICKscript renames emitted to value after the alert node, just before writing to InfluxDB. The message is rendered earlier, inside the alert node. For a deadman rule, use {{ index .Fields "emitted" }} in the message, or drop the value and say “no data for 1m”, which is what the person reading the page needs to know.

Managing databases, users and queries in InfluxDB Admin

The crown icon opens InfluxDB Admin, a thin UI over InfluxQL admin statements. It has three tabs, each mapping to a statement you’d otherwise type in the influx shell:

  • Databases: create and drop databases and retention policies
  • Users: create users and set permissions, filtered by database
  • Queries: the running queries per database with their duration, refreshed every 5 seconds

Creating a database

On the Databases tab, click Create Database. An input row opens above the existing database cards. Type the database name (app_metrics here) and click the green check, which is the Save database button; the X next to it cancels.

Creating the app_metrics database with Create Database in Chronograf InfluxDB Admin
  1. 1Create Database
  2. 2Database name
  3. 3Save database

The UI sends a plain CREATE DATABASE, so the shell sees the result straight away:

influx -execute "SHOW DATABASES"; influx -execute "SHOW RETENTION POLICIES ON app_metrics"

The new database gets one policy, autogen, with a 0s duration, which InfluxDB reads as “keep forever”:

name: databases
name
----
_internal
telegraf
app_metrics
name    duration shardGroupDuration replicaN futureWriteLimit pastWriteLimit default
----    -------- ------------------ -------- ---------------- -------------- -------
autogen 0s       168h0m0s           1        0s               0s             true

That forever default is why the next step matters on any database that takes real traffic.

Adding a retention policy

Still on the Databases tab, hover over the telegraf card and click Add Retention Policy. A new row appears with a Name and Duration field; we used two_weeks and 14d, then clicked Save.

Adding the two_weeks retention policy on the telegraf database in Chronograf InfluxDB Admin
  1. 1Add Retention Policy
  2. 2Name
  3. 3Duration
  4. 4Save

Check that InfluxDB took it:

influx -execute 'SHOW RETENTION POLICIES ON telegraf'

The new policy is there with a 336-hour duration and one-day shard groups, and it’s not the default:

name      duration shardGroupDuration replicaN futureWriteLimit pastWriteLimit default
----      -------- ------------------ -------- ---------------- -------------- -------
autogen   0s       168h0m0s           1        0s               0s             true
two_weeks 336h0m0s 24h0m0s            1        0s               0s             false

Creating the policy does not move any data. Telegraf keeps writing to autogen, which keeps everything forever, until you either make two_weeks the default or set retention_policy in Telegraf’s InfluxDB output. If disk is why you’re here, that second step is the one that saves space.

Creating a read-only user

A data source in Grafana on Ubuntu 26.04 or a reporting script should get a user that can read telegraf and nothing else. On the Users tab, click Create User, fill in the User Name and Password, and click Create.

Create User dialog in Chronograf InfluxDB Admin with the grafana_ro user name and password
  1. 1User Name
  2. 2Password
  3. 3Create

Chronograf then opens the user’s page with a permissions table: one row per database, each with WRITE and READ toggles, all off. Click READ on telegraf. The heading changes to “Database Permissions (unsaved)” and nothing has reached InfluxDB yet. The grant only happens when you click Apply Changes at the top right. Our first attempt closed the page after the toggle and never applied it, and InfluxDB had no grant at all.

Granting grafana_ro READ on the telegraf database, unsaved until Apply Changes is clicked
  1. 1READ on telegraf
  2. 2Apply Changes

After that first attempt, SHOW GRANTS returned an empty table. After clicking Apply Changes, the grant is there:

influx -execute "SHOW USERS"; influx -execute "SHOW GRANTS FOR grafana_ro"

The user is not an admin and holds READ on one database:

user       admin
----       -----
grafana_ro false
database privilege
-------- ---------
telegraf READ

That grant is stored, but whether InfluxDB enforces it depends on one setting, and the default is off.

Why the read-only user can still drop a database

The packaged influxdb.conf ships with auth-enabled commented out, so authentication is off. In that state InfluxDB ignores users entirely. To show it, we logged in as grafana_ro and dropped the database created a minute earlier:

influx -username grafana_ro -password 'ChangeMe-ro-2026' -execute "DROP DATABASE app_metrics"; echo "drop exit: $?"

It worked. The command exited 0 and app_metrics was gone from SHOW DATABASES, so a “read-only” user is only read-only on paper while auth is off:

drop exit: 0
name: databases
name
----
_internal
telegraf

To make the grant mean something, create an admin user first; that’s the account you’ll manage users with once auth is on. Then set auth-enabled = true in the [http] section, confirm the edit took, and restart InfluxDB:

influx -execute "CREATE USER admin WITH PASSWORD 'ChangeMe-admin-2026' WITH ALL PRIVILEGES"
sudo sed -i 's/^  # auth-enabled = false/  auth-enabled = true/' /etc/influxdb/influxdb.conf
grep -n "^  auth-enabled" /etc/influxdb/influxdb.conf
sudo systemctl restart influxdb

The grep must print 336: auth-enabled = true on a stock config. If it prints nothing, the sed missed an edited line and auth is still off, so fix the line by hand before the restart.

We ran the same kind of test again, this time against telegraf itself, followed by the read the user is supposed to have:

influx -username grafana_ro -password 'ChangeMe-ro-2026' -execute "DROP DATABASE telegraf"
influx -username grafana_ro -password 'ChangeMe-ro-2026' -database telegraf -execute "SELECT count(usage_idle) FROM cpu WHERE time > now() - 5m"

Now the drop is refused with “requires admin privilege”, and the read still returns data. Ignore the CLI’s “use <database>” hint; it prints that after any error:

ERR: error authorizing query: grafana_ro not authorized to execute statement 'DROP DATABASE telegraf', requires admin privilege
Warning: It is possible this error is due to not setting a database.
Please set a database with the command "use <database>".
error authorizing query: grafana_ro not authorized to execute statement 'DROP DATABASE telegraf', requires admin privilege
name: cpu
time                count
----                -----
1794467190635823371 87

Turning auth on has a cost you should plan for before the restart. An unauthenticated influx call now fails with “unable to parse authentication credentials”. Within seconds the log of our Telegraf agent filled with Failed to write metric (will be dropped: 401 Unauthorized), and Chronograf’s query proxy for the source answered 403. Those metrics are lost, not queued. So give Telegraf, Kapacitor and the Chronograf source their credentials in the same change window as the restart. The TICK stack hardening article in this series covers those config changes along with HTTPS.

Watching running queries

The Queries tab lists what InfluxDB is running right now, per database, with each query’s duration.

Chronograf InfluxDB Admin Queries tab listing running queries

This is the tab to open when Chronograf feels slow. On our idle box it showed only the SHOW QUERIES calls the tab itself makes, each finishing in microseconds, so a query sitting there for seconds is the cell or rule to look at first.

Keeping dashboards and rules safe on day 2

Everything you built lives in two places that a normal InfluxDB backup doesn’t cover. Dashboards sit in Chronograf’s BoltDB file and the rules sit in Kapacitor’s task store, so losing the VM loses both even if you restore every metric. A nightly API export is cheap insurance. This pulls one dashboard to a file; replace 3 with the ID the list command printed for yours:

curl -s -o mon01-overview-api.json http://localhost:8888/chronograf/v1/dashboards/3 && jq '.cells | length' mon01-overview-api.json

The 4 it prints is the cell count, a quick check that you saved a full dashboard and not an error page. The API returns the bare dashboard object, without the meta wrapper the UI export adds, so keep the UI export for moving dashboards between servers and the API copy for backups.

For the rules, copy the TICKscript part of kapacitor show into a .tick file per task and commit those to Git. The generated scripts carry no dbrp line, so a rebuilt Kapacitor needs it on the command line: kapacitor define <task id> -type stream -tick <file>.tick -dbrp telegraf.autogen, then kapacitor enable <task id>. Before any Chronograf upgrade, stop the service and copy /var/lib/chronograf/chronograf-v1.db; every dashboard, source and Kapacitor connection lives in that one file, so the copy is your rollback if the new version misbehaves.

The one log to watch is /var/log/kapacitor/chronograf-alerts.log, or whichever file your log handler writes. If it goes a week without a single line, test the deadman again by stopping Telegraf. A quiet alert log usually means healthy servers, but once in a while it means a disabled rule or a full disk. If Chronograf is reachable beyond your desk, lock down InfluxDB auth and Chronograf behind HTTPS next; it follows the same Nginx and SSL pattern we use for Grafana. And when the dashboards outgrow Chronograf, the InfluxDB 3, Telegraf and Grafana setup is where the stack goes next. Telegraf itself can feed both, and the Telegraf input plugins for Docker, Nginx and PostgreSQL give these dashboards more than system metrics to draw.

Keep reading

Upgrade Ubuntu 24.04 to Ubuntu 26.04 LTS (Step by Step) Ubuntu Upgrade Ubuntu 24.04 to Ubuntu 26.04 LTS (Step by Step) UFW Firewall Commands with Examples on Ubuntu 26.04 / 24.04 Security UFW Firewall Commands with Examples on Ubuntu 26.04 / 24.04 Backup and Restore Linux Systems with Timeshift Debian Backup and Restore Linux Systems with Timeshift Deploy Collabora Online Office on Ubuntu with Let’s Encrypt SSL Cloud Deploy Collabora Online Office on Ubuntu with Let’s Encrypt SSL Install CyberPanel on Ubuntu 26.04 / 24.04 with Let’s Encrypt Ubuntu Install CyberPanel on Ubuntu 26.04 / 24.04 with Let’s Encrypt Install Apache Cassandra on Rocky Linux 10 / AlmaLinux 10 AlmaLinux Install Apache Cassandra on Rocky Linux 10 / AlmaLinux 10

Leave a Comment

Press ESC to close